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Abstract 

The  primary  objective  of  SI  1333  was  to  develop  innovative  technology  to  increase  the  speed 
and  repeatability  with  which  reef  plots  can  be  mapped  and  inventoried.  Specifically,  we  used 
underwater  images  to  create  landscape  (2D)  mosaics  of  reef  plots  in  a  highly  automated  way. 
The  innovative  aspect  of  the  mosaicing  technology  is  that  the  images  provide  both  landscape- 
level  (meter-scale)  maps  and  high-resolution  (sub-millimeter)  images  of  individual  coral 
colonies.  Users  can  collect  imagery  for  areas  of  several  hundred  square  meters  in  under  an  hour 
of  in-water  dive  time  to  create  mosaics  that  provide  information  on  coral  colony  health  and 
small-scale  competitive  interactions.  The  mosaic  products  are  useful  for  extracting  ecological 
indicators  of  reef  health  and  for  damage  assessment;  they  also  have  excellent  archive  potential 
and  are  superior  tools  for  tracking  changes  over  time. 

As  secondary  goals,  SI  1333  explored  two  techniques  to  assist  or  automate  classification  of 
underwater  imagery:  (i)  3D  reconstruction  of  specific  reef  features  and  (ii)  high  resolution 
multispectral  imaging.  A  3D  tool  was  developed  that  allows  users  to  visualize  and  measure 
topographic  structure  and  heights  of  single  objects,  such  as  coral  colonies,  from  landscape 
mosaics  created  using  stereo  imagery.  In  addition,  an  automated  seabed  classification  algorithm 
based  on  texture  analysis  of  narrow  spectral  band  images  was  identified  that  can  reliably  segment 
corals,  algae  and  the  non-photosynthetic  background.  These  results  suggest  that  high  spectral 
resolution  combined  with  texture-based  image  segmentation  may  be  an  optimal  methodology  for 
automated  classification  of  underwater  coral  reef  imagery. 

Objectives 

The  development  of  underwater  landscape  mosaicing  capability  addresses  an  emerging  need  to 
augment  diver  surveys  in  an  effort  to  efficiently  inventory  and  monitor  large  areas  of  DoD-held 
coral  reef  resources.  Mapping  is  a  crucial  component  of  establishing  baseline  environmental  data 
and  a  primary  goal  of  SON  CSSON-03-02.  For  coral  reefs,  successful  and  legally  defensible 
monitoring  of  reef  condition  requires  estimates  of  basic  ecological  parameters,  such  as  live  coral 
cover,  species  diversity,  and  mortality/recruitment  rates.  Presently,  such  parameters  are  typically 
measured  during  field  surveys  using  trained  divers.  Airborne  or  satellite-based  remotely  sensed 
data  are  not  currently  able  to  reliably  quantify  coral  condition  at  the  required  level  of  detail 
(Mumby  et  al.  1998).  The  most  appropriate  strategy  for  reef  monitoring  is  detailed  analysis  of 
meter-scale  plots  at  high  spatial  resolution. 

The  aim  of  Project  SI  1333  was  to  develop  technology  that  will  increase  the  speed  and 
repeatability  with  which  reef  plots  can  be  mapped  and  inventoried.  Specifically,  we  are  using 
underwater  video  to  create  landscape  mosaics  of  reef  plots  in  a  highly  automated  way.  Our 
objective  was  is  to  construct  spatially  accurate  landscape  (2D)  video  mosaics  of  reef  plots  and  to 
extract  meaningful  ecological  indices  of  reef  condition  from  these  mosaics.  This  landscape 
mosaicing  technology  offers  numerous  advantages  over  traditional,  diver-based  video  transects 
(ID)  for  coral  reef  mapping  and  monitoring.  Landscape  mosaics  produce  single,  spatially 
accurate,  plot-scale,  high-resolution  images  that  can  be  georeferenced. 
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As  an  extension  to  our  original  funded  proposal  we  expanded  our  primary  objective  to  include 
expanded  mosaicing  capabilities  and  streamlined  processing.  The  specific  goals  of  this  project 
extension  were  to  1)  reduce  the  impact  of  sunflickering  interference  on  mosaic  processing,  2) 
improve  image  blending  to  create  seamless  mosaics,  3)  integrate  a  heading  sensor  for  use  in  high 
relief  settings  and  4)  integrate  a  still  camera  for  increased  benthic  resolution  of  landscape 
mosaics.  The  resulting  landscape  mosaics  allow  users  to  extract  increased  reef  health  information 
in  complex  reef  habitats  and  provide  a  better  overall  product  with  increased  value  to  coral  reef 
monitoring  users  for  change  detection  analyses.  These  are  essential  capabilities  for  the  legally 
mandated  environmental  documentation  necessary  for  conducting  military  operations  and  could 
provide  decision-makers  with  crucial  information  necessary  to  maintain  compliance  with 
relevant  statutes,  regulations,  and  executive  orders. 

In  addition  to  the  primary  goal  of  developing  an  underwater  landscape  mosaicing  capability  for 
use  in  coral  reef  monitoring  and  mapping  applications,  we  have  explored  two  techniques  as 
secondary  project  goals  to  assist  or  automate  classification  of  underwater  imagery:  (i)  3D 
reconstruction  of  specific  reef  features  and  (ii)  underwater  multispectral  imaging. 

The  objective  of  the  3D  effort  was  to  develop  a  3D  reconstruction  tool  that  allows  a  user  to 
visualize  and  measure  topographic  structure  and  heights  of  single  objects,  such  as  coral  colonies, 
from  2D  mosaics  created  from  stereo  imagery.  Our  work  on  3D  reconstruction  of  reef  features 
investigated  two  types  of  approaches:  1)  an  optical  flow-based  method  applied  to  monocular 
video  sequence  for  the  dense  estimation  of  a  3D  map,  and  2)  a  feature-based  technique  allowing 
3D  reconstruction  of  single  objects  from  stereo  imagery.  We  determined  that  the  second 
approach  was  more  suitable  for  developing  a  tool  for  non-expert  use  in  coral  reef  monitoring 
applications.  The  materials  and  methods  as  well  as  results  of  this  secondary  goal  are  presented 
in  Section  B  of  this  report  and  can  be  read  as  a  standalone  report. 

The  objectives  of  the  multispectral  imaging  effort  were  to  build  and  deploy  an  underwater 
multispectral  camera  to  test  whether  the  spectral  bands  suggested  in  the  literature  (Holden  and 
LeDrew  1998;  Holden  and  LeDrew  1999;  Clark  et  al.  2000;  Hochberg  and  Atkinson  2000; 
Holden  and  LeDrew  2001;  Holden  and  LeDrew  2002;  Hochberg  and  Atkinson  2003;  Hochberg 
et  al.  2003)  could  be  used  to  automate  the  classification  of  underwater  imagery  from  coral  reef 
environments.  A  further  objective,  which  developed  during  the  course  of  the  research,  was  to  test 
whether  simple  texture  metrics  in  combination  with  narrow-band  spectral  imagery  could 
automatically  classify  basic  bottom  cover  types  associated  with  coral  reefs,  such  as  coral,  algae, 
and  sand.  The  materials  and  methods  as  well  as  results  of  this  secondary  goal  are  presented  in 
Section  C  of  this  report  and  can  be  read  as  a  standalone  report. 

Background 

Recent  declines  in  coral  reefs  across  the  globe  underscore  the  need  for  new  scientific  tools  to 
better  understand  ecological  patterns  and  rates  of  change.  Of  immediate  interest  to  the  Dept,  of 
Defense  are  mesoscale  measurements  that  detect  changes  within  reef  systems  by  monitoring 
meter-scale  plots. 
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The  state-of-the-art  in  mesoscale  coral  reef  assessment  consists  of  a  combination  of  diver-based 
and  image-based  measurements.  The  "gold  standard"  of  reef  monitoring,  in  terms  of  obtaining 
maximum  level  of  detail,  is  assessment  by  expert  scientific  divers.  Diver-based  assessments  are, 
however,  often  time  consuming  in  the  water,  require  a  high  level  of  individual  training,  and 
provide  a  limited  permanent  record  of  the  state-of-the-reef  at  the  time  of  the  survey. 
Technological  aids,  in  the  form  of  image-based  techniques  such  as  photo  quadrats  and  ID  strip 
mosaics,  have  been  adopted  to  complement  diver-based  assessments.  Digital  photographs  or 
video  images  of  small  plots  (<  lm2)  are  commonly  used  to  measure  benthic  cover,  species 
diversity,  and  coral  condition  within  reef  habitats.  Similarly,  photographs  of  permanent  plots  and 
coral  colonies  marked  with  metal  stakes  or  nails  are  used  to  track  changes  over  time.  Video 
surveys  of  the  reef  benthos  can  be  analyzed  by  extracting  individual  frames,  which  are  then 
treated  as  photo-quadrats  or  by  stitching  the  frames  together  into  a  1 -dimensional  ‘strip’  mosaic, 
which  provides  a  limited  landscape  view  of  the  bottom. 

At  least  one  coral  reef  monitoring  program,  the  Coral  Reef  Monitoring  Program  of  the  Florida 
Keys  National  Marine  Sanctuary,  is  using  strip-mosaic  software  (. RavenView  by  Observera)  to 
provide  expanded  views  of  the  bottom  and  extract  ecological  information  (Jaap  et  al.,  2002).  In 
this  program,  divers  collect  video  and  the  imagery  is  stitched  together  to  provide  a  strip  mosaic 
of  22  m  x  40  cm.  The  video  mosaics  are  then  analyzed  to  extract  information  on  the  benthic 
cover  of  reef  organisms.  Because  these  images  are  collected  at  a  short  distance  from  the  bottom 
(generally  40  cm)  and  only  partial  views  of  larger  organisms  are  obtained,  size-estimation  is 
limited  to  the  smallest  coral  colonies  that  are  completely  imaged  within  a  frame  or  a  transect. 
Moreover,  the  lack  of  image  registration  limits  spatial  accuracy  and  precludes  the  estimation  of 
spatial  patterns  within  plots. 

Although  photo-quadrats  and  video  surveys  improve  the  efficiency  of  diver-based  surveys  by 
shifting  expert  analysis  and  species  identifications  from  the  field  to  the  lab,  these  image  based 
monitoring  tools  have  many  limitations.  In  particular,  photo-quadrats  and  ID  strip  mosaics  have 
limited  footprints,  thereby  restricting  the  assessment  area  and  often  requiring  placement  of 
numerous  markers  and  tags  for  repeated  surveys  of  the  plots  or  colonies  of  interest. 

Work  in  terrestrial  systems  has  successfully  implemented  image -based  mosaicing  covering  large 
areas.  State-of-the-art  aerial  photogrammetric  mapping  methods  using  digital  technology  are 
highly  automated  and  efficient.  These  techniques  are,  however,  not  applicable  to  underwater 
imaging  because  they  rely  on  input  from  Global  Positioning  Systems  (GPS),  which  do  not 
operate  underwater.  Aerial  imaging  algorithms  can  be  modified  for  underwater  mosaicing 
through  use  of  acoustic  networks  to  provide  navigation  information.  This  approach  has  been  used 
for  well-funded  deep-sea  projects  (e.g.  archaeology  with  the  WHOI  Jason  submersible;  Foley 
and  Mindell,  2002);  it  is,  however,  not  suitable  for  routine  reef  monitoring  because  of  the 
expense  of  the  equipment  and  the  time  required  to  install  a  sonar  network,  combined  with  the 
number  of  sites  that  are  normally  surveyed  in  a  routine  monitoring  program. 

Image  panorama  software  that  blends  components  of  several  images  taken  from  a  stationary 
point  (such  as  PTgui  by  NHIS,  Panorama  Composer  by  FirmTool  or  Cool360  by  ULead)  is 
universally  unsuited  to  coral  reef  mapping  applications  because  the  algorithms  cannot  handle 
movement  of  the  camera  over  an  area  of  interest.  Algorithms  designed  for  this  type  of 
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application  assume  all  images  are  acquired  from  the  same  point,  with  camera  rotation  being  the 
variable  between  images.  Due  to  the  attenuation  of  light  by  water,  underwater  mapping  images 
must  be  acquired  by  moving  the  camera  between  images.  The  resulting  images  vary  not  only  in 
rotation  but  also  significantly  in  translation.  Algorithms  that  assume  no  translation  fail  with  this 
type  of  data  input. 

Many  manual  photogrammetric  systems  that  allow  users  to  define  common  points  between 
images  are  available,  and  some  have  been  used  for  underwater  2-D  and  even  3-D  applications 
(Gifford,  1997;  Courtney  et  al.,  2006).  These  systems  produce  good  results,  but  require  intensive 
user  input  and  are  not  practical  for  producing  mosaics  comprised  of  several  hundred  to  over  a 
thousand  images  on  a  routine  basis. 

2D  landscape  mosaic  technology,  as  developed  during  this  project,  presents  significant  advances 
in  state-of-the-art  capabilities  for  reef  mapping  and  monitoring.  The  most  significant  advances 
result  from  the  fact  that  large  areas  can  now  be  imaged  at  high  spatial  resolution  (on  the  order  of 
400  m2  at  1-2  mm/pixel),  resulting  in  spatially  accurate,  landscape  views  of  the  bottom  that  were 
previously  unobtainable.  These  landscape  mosaics  will  be  useful  for  DoD  reef  monitoring 
requirements  and  open  doors  for  new  applications  in  reef  mapping  and  change  detection. 

Project  SI  1333  is  based  on  the  premise  that  the  use  of  large-scale,  2D  mosaic  images  of  reef 
plots  can  circumvent  the  limitations  of  diver  transects,  photo-quadrats,  and  ID  strip  mosaics, 
while  simultaneously  maintaining  the  strengths  of  the  diver  approach  for  the  purposes  of  coral 
reef  monitoring  by  the  Navy. 

Secondary  goal:  3D  reconstructions 

One  of  the  defining  and  most  important  attributes  of  coral  reef  communities  is  the  3D  structure 
provided  by  stony  corals.  Within  coral  reefs,  the  topographical  complexity  created  by  corals 
provides  essential  habitat  for  the  multitude  of  organisms  that  form  part  of  one  of  the  most  diverse 
and  productive  ecosystems  in  the  planet.  In  addition  to  supporting  these  diverse  communities,  the 
structures  created  by  corals  provide  shoreline  protection  that  prevents  storm  damage  and  coastal 
erosion. 

Because  of  the  key  role  of  habitat  complexity  in  reef  health,  monitoring  tools  able  to  document 
3D  topography  can  provide  valuable  information  on  reef  structure  and  function.  These 
parameters  are  not  commonly  obtained  by  standard  monitoring  methodologies,  which  measure 
coral  cover  as  the  proportion  of  the  bottom  occupied  by  corals  in  a  planar  view. 

Previous  research  has  demonstrated  that  a  3D  approach  provides  a  more  realistic  quantification 
of  coral  structure  available  as  essential  habitat  and,  when  combined  with  estimates  of  living  coral 
tissue,  quantifies  the  amount  of  live  coral  tissue  available  (Fisher  et  al.  2007).  In  colonial 
organisms  like  corals,  most  physiological  processes,  such  as  calcification,  growth,  mortality,  and 
fecundity,  are  directly  linked  to  the  surface  area  of  coral  colonies.  Accordingly,  the  estimation  of 
colony  size  based  on  single  planar  dimensions  (e.g.,  maximum  diameter,  projected  surface  area) 
commonly  underestimates  the  amount  of  coral  tissue. 
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Although  2D  video  mosaics  are  effective  tools  to  measure  the  size  of  coral  colonies  based  on 
maximum  diameter,  the  added  capability  of  extracting  colony  height  measurements  from  stereo 
views  can  provide  all  of  the  information  needed  to  calculate  both  the  volume  and  the  surface  area 
of  coral  colonies.  Most  coral  colonies  approximate  hemispherical  shapes  and,  therefore,  by 
measuring  colony  diameter  and  colony  height  and  incorporating  these  measurements  into 
standard  geometric  formulae,  both  the  volume  of  the  skeletal  structures  and  the  surface  area  of 
live  tissue  available  can  be  easily  calculated  (Fisher  et  al.  2007). 

Secondary  goal:  Multispectral  Imaging 

Underwater  video  and  still  imagery  are  useful  tools  for  coral  reef  monitoring  programs  because 
they  speed  up  the  data  acquisition  process,  allowing  more  sites  to  be  visited  for  a  given  allotment 
of  field  time  and  because  imagery  can  be  acquired  by  divers  with  minimal  biological  training. 
The  disadvantage  of  an  image -based  approach  is  the  time  required  to  extract  useful  ecological 
information  from  the  underwater  imagery.  Current  state-of-the-art  techniques  require  identifying 
organisms  and  substrate  through  the  use  of  either  random  points  placed  on  each  image  (point 
counting)  or  tracing  individual  objects.  Such  manual  processing  of  each  video/still  frame  is  not 
only  labor  intensive,  but  also  requires  an  analyst  able  to  identify  coral  reef  organisms. 

Extracting  data  from  underwater  imagery  is  thus  a  major  portion  of  the  budget  for  coral  reef 
monitoring  programs.  Any  procedure  that  automates  or  streamlines  part  of  the  image  analysis 
process  could,  therefore,  reduce  costs  associated  with  monitoring  reef  resources.  Increasing  the 
efficiency  of  information  extraction  from  underwater  imagery  is  the  DoD  regulatory  problem 
that  this  technology  addresses.  Several  software  packages  designed  to  streamline  the  point 
counting  process  have  been  released  in  the  past  decade1  (Kohler  and  Gill  2006).  These  packages 
do  not  actually  automate  the  extraction  of  data;  however,  they  simply  facilitate  manual  data  entry 
and  storage. 

Color  matching,  laser  line  scan  imagery,  and  texture  segmentation  are  three  approaches  that  have 
been  explored  previously  in  attempts  to  automate  the  classification  of  underwater  imagery. 
Classification  using  color  segmentation  of  standard  broad-band  imagery  has  not  been  successful 
(Bernhardt  and  Griffmg  2001).  Interactive  classification  using  color  matching  was  more 
successful  (Bernhardt  and  Griffmg  2001),  but  very  time  consuming.  Underwater  laser  line  scan 
imagery  showed  potential  for  automatically  classifying  the  seabed  in  coral  reef  environments 
(Mazel  et  al.  2003),  but  laser  line  scan  instruments  are  currently  far  too  expensive  for  routine 
coral  reef  monitoring  projects.  Recently,  several  groups  have  initiated  efforts  to  automate 
classification  of  underwater  imagery  using  image  texture  in  combination  with  neural  network 
(Konotchick  et  al.  2006;  Pizarro  et  al.  2006)  or  support  vector  machine  (Mehta  et  al.  2007) 
classifiers.  Such  an  approach  may  prove  fruitful,  but  requires  many  training  images  (or  portions 
of  images)  and  significant  computing  resources. 

Several  studies  have  shown  that,  at  least  in  principle,  the  hyperspectral  reflectance  of  coral  reef 
organisms  can  be  used  to  discriminate  functional  groups  (e.g.  corals,  algae,  and  sediment) 
(Hochberg  and  Atkinson  2000;  Hochberg  and  Atkinson  2003;  Hochberg  et  al.  2003),  identify  the 
"health"  of  coral  tissue  (live,  dead,  bleached)  (Holden  and  LeDrew  1998;  Holden  and  LeDrew 


1  For  example  PointCount99  (http://www.cofc.edu/~coral/pc99/pc99.htm)  and  Vidana 
(http://www.projects.ex.ac.Uk/msel/vidana/j 
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1999;  Clark  et  al.  2000;  Holden  and  LeDrew  2001;  Holden  and  LeDrew  2002),  and  map  coral 
reef  communities  (Louchard  et  al.  2003;  Mobley  et  al.  2004).  The  major  contribution  of  these 
efforts  has  been  to  suggest  spectral  bands  that  might  be  optimum  for  mapping  and  monitoring 
coral  reefs  from  satellite  or  airborne  imagery. 

A  first  step  in  our  effort  in  multispectral  imaging  was  to  test  whether  the  spectral  bands 
suggested  in  (Holden  and  LeDrew  1998;  Holden  and  LeDrew  1999;  Clark  et  al.  2000;  Hochberg 
and  Atkinson  2000;  Holden  and  LeDrew  2001;  Holden  and  LeDrew  2002;  Hochberg  and 
Atkinson  2003;  Hochberg  et  al.  2003)  could  be  used  to  automate  the  classification  of  underwater 
imagery  (Figure  Cl).  A  computer-controlled  underwater  camera  with  a  filter  wheel  that  holds  six 
narrow-band  (10  nm)  interference  filters  was  used  to  acquire  multispectral  images  both  in  salt¬ 
water  tanks  at  the  University  of  Miami  and  on  coral  reefs  in  the  Bahamas  and  Florida  Keys. 

Attempts  to  classify  the  multispectral  images  using  the  algorithms  suggested  by  (Holden  and 
LeDrew  1998;  Holden  and  LeDrew  1999;  Clark  et  al.  2000;  Hochberg  and  Atkinson  2000; 
Holden  and  LeDrew  2001;  Holden  and  LeDrew  2002;  Hochberg  and  Atkinson  2003;  Hochberg 
et  al.  2003)  were  not  successful.  As  a  byproduct  of  those  experiments,  however,  it  was  noted  that 
spectral  bands  near  545  and  570  nm  were  repeatedly  identified  as  significant  in  discriminating 
live  coral  from  other  spectra.  A  ratio  of  the  two  bands  chosen  in  that  spectral  range  (546  and  568 
nm)  was  able  to  segment  coral  and  algae,  together,  from  other  objects  but  was  not  able  to 
separate  coral  from  algae. 

Since  segmenting  coral  and  algae,  together  as  one  class,  is  not  especially  useful  for  ecological 
purposes,  image  texture  analysis  methods  were  investigated  to  determine  if  a  simple  spatial 
technique  could  assist  the  spectral  processing  by  separating  coral  and  algae  into  two  distinct 
classes. 

A  second  step,  we  then  investigated  whether  simple  texture  metrics  in  combination  with  narrow- 
band  spectral  imagery  could  automatically  classify  basic  bottom  cover  types  associated  with 
coral  reefs,  specifically:  coral,  algae,  and  "other".  The  general  approach  was  first  to  segment 
coral  and  algae  from  the  background  using  the  normalized  difference  ratio  of  images  at  546  and 
568  nm,  then  to  segment  coral  from  algae  using  texture  metrics  computed  from  grey  level  co¬ 
occurrence  matrices. 


Methods,  Results,  and  Conclusions 

Methods,  results  and  conclusions  pertaining  to  the  primary  and  secondary  goals  of  project  SI 
1333  are  presented  in  three  independent  sections.  Section  A  presents  information  relevant  to  the 
primary  goal:  creation  and  analysis  of  landscape  (2D)  mosaics  for  coral  reef  mapping  and 
monitoring.  Sections  B  and  C  summarize  research  associated  with  the  secondary  goals,  namely 
single  object  3D  reconstruction  (Section  B)  and  underwater  multispectral  imaging  (Section  C)  as 
tools  to  assist  mosaic  classification  and  analysis. 
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Section  A:  Landscape  Mosaics 

Our  research  on  landscape  (2D)  mosaics  is  based  on  the  premise  that  use  of  large-scale,  2D 
images  of  reef  plots  can  circumvent  the  limitations  of  current  state-of-the-art  methods  in  coral 
reef  monitoring  (i.e.  diver  transects,  photo-quadrats,  strip  mosaics),  while  simultaneously 
maintaining  the  strengths  of  a  diver-based  approach.  We  have  developed  techniques  to  construct 
spatially  accurate  mosaics  ~  20  m  x  20  m  in  extent  with  millimeter-scale  resolution. 

Our  mosaic  algorithm  involves  three  major  steps  (1)  estimating  image  motion  by  matching 
sequential  video  frames,  (2)  global  alignment  to  match  non-sequential  frames,  and  (3)  blending 
to  render  the  mosaic  image.  The  first  generation  mosaics  provided  centimeter-scale  resolution, 
but  lacked  sufficient  detail  needed  for  species-level  identification  of  benthic  organisms.  In  2007 
the  mosaicing  system  was  upgraded  by  integrating  a  high-resolution  still  camera  with  the  video 
acquisition  system.  This  ‘second  generation’  mosaicing  system  produces  mosaics  with  mm- 
scale  resolution. 

The  resulting  mosaics  excel  in  several  areas  limited  by  poor  performance  using  traditional,  state 
of-the-art,  reef  monitoring  techniques: 

•  Mosaics  provide  a  landscape  view  of  coral  reefs  that  has  previously  been  unobtainable. 

•  Mosaics  are  superior  tools  to  track  patterns  of  change  over  time. 

•  Mosaics  have  a  spatial  accuracy  on  the  order  of  centimeters  to  millimeters. 

As  a  consequence  of  the  capabilities  above,  mosaics  are  superior  tools  for  monitoring  disease, 
bleaching,  and  partial  mortality— all  important  indicators  of  reef  health. 

Al.  Methods 

The  basic  algorithm  used  in  this  project  for  creating  underwater  mosaics  stems  from  work  by 
Gracias  and  Santos- Victor  (2000;  2001).  The  basic  algorithm  involves  three  main  steps.  The  first 
step  computes  the  sequential  estimation  of  image  motion,  using  a  subset  of  the  images  captured 
by  the  video  camera.  The  set  of  resulting  consecutive  homographies  (i.e.,  coordinate  mapping 
between  two  image  projections  of  the  same  3D  plane)  is  cascaded  to  infer  the  approximate 
trajectory  of  the  camera.  The  trajectory  information  is  then  used  to  predict  the  areas  where  there 
will  be  image  overlap  from  non-consecutive  images  (i.e.,  neighboring  video  strips).  To  reduce 
the  algorithmic  complexity  and  memory  requirements,  a  set  of  key  frames  are  selected  based  on 
an  image  superposition  criterion  (typically  65%-80%).  Only  such  key  frames  are  used  in  the 
following  optimization  steps. 

In  the  second  step,  a  global  alignment  is  performed  where  the  overall  camera  trajectory  is  refined 
by  performing  the  following:  (1)  point  correspondences  are  established  between  non-adjacent 
pairs  of  images  that  present  enough  overlap;  and  (2)  the  trajectory  is  updated  by  searching  for  the 
set  of  trajectory  parameters  that  minimizes  the  overall  sum  of  distances  in  the  point  matches.  A 
least  squares  criterion  is  minimized  using  a  non-linear  optimization  algorithm. 
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The  final  step  of  the  mosaicing  process  consists  of  blending  the  images  (i.e.,  choosing 
representative  pixels  from  the  spatially  registered  images  to  render  the  mosaic  image).  The 
mosaic  is  created  by  choosing  the  contributing  pixels  that  are  closest  to  the  center  of  their 
frames.  This  method  preserves  the  texture  of  the  seafloor  and  reduces  artifacts  due  to  registration 
misalignments  of  3D  structure. 

The  method  for  mosaic  creation  was  initially  tailored  for  processing  video  acquired  by  a  single 
camera,  either  deployed  by  a  diver  or  by  an  underwater  vehicle.  The  method  was  further 
extended  to  deal  with  a  two-camera  configuration,  comprising  a  video  and  a  high-resolution  still 
camera.  The  extended  method  performs  the  registration  of  stills  against  the  video  frames  in  order 
to  find  the  adequate  placement  of  the  stills  in  the  mosaic.  Upgrades  and  extensions  to  the  original 
algorithm  are  described  in  Section  A2 

A2.  Results  and  Discussion 

Major  accomplishments  for  project  SI  1333,  as  summarized  by  year,  include  the  following:  Year 
1  (June-Dee  2003)  focused  on  system  development  (camera  resolution  improvement  and  better 
positioning  capability)  and  collection  of  initial  data  sets  at  Florida  and  Andros  sites.  Year  2 
(2004),  was  devoted  to  creating  2D  mosaics  of  reef  plots,  assessing  the  geometric  accuracy  of  the 
mosaics  and  performing  ecological  analysis  of  the  plots.  Accomplishments  in  Year  3  (2005) 
focused  on  exploratory  enhancements  of  the  2D  mosaicing  algorithm,  groundtruthing  ecological 
analyses  from  mosaic  products,  and  applying  mosaics  to  non-traditional  reef  monitoring 
applications.  In  Year  4  (2006),  2D  video  mosaic  technology  was  applied  to  different  reef 
monitoring  needs  such  as  hurricane  damage  assessment,  grounding  and  coral  bleaching  recovery. 
Year  5  (2007)  focused  on  upgrading  the  mosaicing  imaging  platform  and  integrating  new 
features  into  the  2D  mosaicing  software;  the  new  platform  with  enhanced  capability  is  referred  to 
as  a  ‘second  generation’  system,  which  produces  ‘high  resolution’  mosaics.  In  addition,  user- 
friendly  software  for  2D  mosaic  creation  and  extraction  of  ecological  indices  was  developed  in 
Year  5.  Work  during  Year  6  (2008)  focused  on  continued  development  and  utilization  of  the 
second-generation  mosaic  technology  for  coral  reef  monitoring  applications,  development  and 
integration  of  enhanced  capabilities  in  the  mosaicing  software  deliverable,  and  continuing 
collaborations  with  other  government  and  scientific  users.  Year  7  (2009)  has  focused  on 
continued  research  with  governmental  and  educational  collaborators  as  well  as  a  finalization  of 
the  mosaic  module  deliverable  and  manual.  Below  is  a  summary  of  major  accomplishments 
throughout  the  project.  These  accomplishments  are  subdivided  with  respect  to  1)  mosaic 
algorithm  development,  2)  mosaic  products,  3)  analysis  of  mosaic  products,  4)  applications  of 
landscape  mosaic  technology  to  reef  monitoring  and  mapping,  5)  collaborations  with 
governmental  and  educational  users,  and  6)  project  deliverables. 

A2.1  Mosaic  algorithm  development 

A2.1.1  Initial  mosaicing  capability 

The  first  mosaics  constructed  in  project  SI  1333  were  based  on  techniques  outlined  by 
Negahdaripour  et  al  (1998).  A  low-resolution,  real-time  algorithm  was  used  to  estimate  the 
image  motion  and  to  create  a  live  preview  of  small  areas  of  the  benthos  (in  the  order  of  tens  of 
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square  meters).  This  algorithm  was  built  for  fast  operation,  and  was  targeted  at  real-time  image 
motion  estimation  for  ROV  position  control  and  station-keeping. 

The  initial  mosaicing  capability  was  restricted  to  low-resolution  black  and  white  images  of  short 
duration  such  as  those  shown  in  Figure  Al.  Although  this  was  a  necessary  first  step  in  creating 
useful  mosaics  for  reef  resource  mapping  and  monitoring,  black  and  white  video  images  did  not 
provide  the  benthic  detail  necessary  for  ecological  analysis.  Therefore,  our  cameras  were 
upgraded  to  color  video  cameras  with  single  mega-pixel  frames  (~1 000x800),  rather  than  the 
standard  640x480  interlaced  video.  The  difference  in  benthic  identification  ability  is  shown  in 
figure  Al.  Objects  that  were  unidentifiable,  such  as  the  brick  in  the  top  portion  of  the  image,  are 
became  clear  using  megapixel  color  imagery. 


Figure  Al.  Early  mosaicing  results.  Low-resolution  mosaic  (a)  constructed  with  the  real-time  vision  system  of 
our  ROV  in  comparison  with  a  high-resolution  color  mosaic  generated  offline  (b).  The  red  line  in  (a)  shows 
the  trajectory. 


The  Negahdaripour  et  al  (1998)  algorithm  had  difficulty  creating  a  2D  “landscape”  view  over 
complex  seabeds,  so  the  first  mosaics  for  this  project  had  to  be  assembled  as  a  series  of  strips 
covering  the  area  of  interest  (Figure  A2). 
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Figure  A2  .  Mosaics  of  the  4  tracks  of  Andros  data,  collected  over  a  3mx3m  grid.  Misalignments  in  certain 
parts  of  each  view  are  directly  tied  to  the  3D  nature  of  the  terrain,  as  current  mosaicing  algorithms  assume  a 
flat  terrain  model. 
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Image  motion  estimation  in  the  early  mosaics  was  based  in  optical-flow  techniques,  which  are 
not  able  to  deal  with  departures  from  the  underlying  assumption  of  a  flat  sea  floor.  Using  this 
technique,  image  motion  estimation  and  mosaic  previews  were  being  made  concurrently.  The 
motion  estimation  used  the  mosaic  as  a  reference  map.  As  a  result,  any  misalignments  created  by 
improper  image  motion  estimation  were  immediately  incorporated  into  the  resulting  mosaic, 
which  would  further  hinder  the  motion  estimation.  This  process,  although  fast,  did  not  produce 
the  geometrically  accurate  results  needed  for  repeated  mosaic  surveys  of  the  same  area.  This  in 
combination  with  the  of  low  resolution  of  the  real-time  image  processing  made  the  approach 
unsuited  for  mosaicing  areas  larger  than  tens  of  square  meters. 


A2.1.2  Refined  algorithm 

In  2004,  we  implemented  a  new  technique  for  off-line  creation  of  2D  mosaics,  which  enabled 
creation  of  mosaics  with  extended  area  coverage.  This  capability  was  developed  by  building  on 
the  Ph.D.  work  of  project  member  Nuno  Gracias  (Gracias  et  al  2003).  The  technique  involves 
three  processing  phases,  as  follows: 

i.  Sequential  Estimation.  In  this  step,  image  motion  between  time  consecutive  video  frames  is 

estimated  by  finding  corresponding  image  points  across  pairs  of  images.  This  results  in  a 
set  of  ordered  image-to-image  mappings.  The  correspondences  are  found  using  a  mixture 
of  Scale  Invariant  Feature  Transform  (SIFT)  (Lowe,  2004)  and  the  Harris  comer  detector 
(Harris  and  Stephens,  1988)  with  normalized  cross-correlation.  Robust  estimation 
techniques  are  used  to  filter  out  wrongly  matched  pairs  of  points,  which  may  appear  due 
to  strong  3D  or  to  moving  algae  or  fish.  These  techniques  make  the  newer  mosaicing 
capability  much  more  resilient  to  a  non-flat  benthos.  The  mappings  are  cascaded  in  order 
to  infer  the  approximate  camera  trajectory.  The  accuracy  of  this  step  is  limited  because 
small  unavoidable  errors  in  motion  estimation  lead  to  growth  of  errors  in  camera 
trajectory.  This  feature-based  approach  to  estimating  image  motion  replaced  the  flow- 
based  motion  estimation  technique  described  in  our  original  proposal. 

ii.  Alignment.  In  this  step,  an  estimate  of  the  camera  trajectory  is  used  to  predict  areas  of  image 

overlap  resulting  from  loops.  When  such  regions  are  found,  additional  constraints  on  the 
camera  trajectory  can  be  imposed,  thereby  improving  the  spatial  correctness  of  the  final 
mosaic.  The  algorithm  performs  a  batch  optimization  on  the  global  set  of  motion 
constraints.  (Gracias  et  al  2003,  Lirman  et  al  2007). 

Hi.  Rendering.  The  final  mosaic  image  is  created  by  warping  and  blending  the  original  video 
frames,  using  the  optimization  result  of  the  previous  step  (Gracias  et  al  2006,  Gracias  et 
al  2009). 

Figure  A3  shows  results  using  the  Gracias  et  al  2003  mosaicing  algorithm.  To  illustrate  the  need 
for  the  global  estimation  phase,  the  upper  image  was  rendered  using  only  sequential  estimation 
(first  phase  only),  while  the  lower  one  was  created  with  the  full  algorithm.  The  error 
accumulation  is  visible  on  the  repetitive  pattern  (marked  by  the  dotted  lines),  which  corresponds 
to  the  same  area  of  the  seabed:  a  single  cross  over  point  of  two  cables. 
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Figure  A3.  Mosaic  creation  example  using  Andros  data.  Top:  Mosaic  created  using  only  sequential  motion 
estimation.  Bottom:  Mosaic  incorporating  global  alignment.  The  circled  areas  designate  a  single  cross-over 
point  of  two  cables.  Note  that  this  single  point  is  repeated  several  times  in  the  top,  unaligned  mosaic  due  to 
accumulated  error. 
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The  implementation  of  the  revised  mosaicing  approach  allowed  for  mosaic  creation  of  color 
imagery  in  a  single  landscape  image  that  was  previously  unattainable. 

A2.1.3Enhancements  to  the  2D  Mosaicing  Algorithm 

Considerable  effort  from  2004-2009  was  devoted  to  enhancing  the  Gracias  et  al  2003  2D 
mosaicing  algorithm.  An  important  limitation  to  this  algorithm  is  an  underlying  assumption  that 
the  area  being  imaged  is  dominantly  flat.  We  were  able  to  mosaic  the  sea  floor  for  moderate 
departures  of  this  assumption  such  as  areas  where  the  height  of  the  benthic  structures  is  roughly 
less  than  %  of  the  distance  to  the  sea  floor.  However  3D  content  can  lead  to  registration 
inaccuracies  that  degrade  the  appearance  of  the  resulting  mosaics  and  hinder  analysis.  To  better 
deal  with  topography,  improve  the  quality  of  the  mosaic  products  and  increase  processing  speed, 
a  variety  of  enhancements  to  the  2D  mosaicing  technique  were  developed,  as  described  below. 

A2. 1.3.1  Topography:  Combining  Different  Altitude  Sequences 

In  order  to  make  more  accurate  mosaics  of  areas  with  topographic  relief,  we  devised  and  tested 
two  different  methods.  One  method  creates  a  higher  altitude  mosaic  against  which  lower  altitude 
images  are  directly  registered.  When  using  2D  registration  methods,  higher  altitude  images  are 
more  reliably  registered.  This  is  due  to  the  smaller  topography  variations,  scaled  by  the  distance 
to  the  scene.  Conversely,  lower  altitude  images  provide  higher  resolution  views,  but  are  more 
difficult  to  register.  Combining  both  can  significantly  increase  the  robustness  and  efficiency  of 
the  mosaicing  process,  since  higher  altitude  sequences  can  be  used  to  guide  the  registration  of  the 
lower  ones. 

The  second  method  for  dealing  with  topography  uses  the  displacement  of  the  images  on  a 
common  spatial  reference  frame.  The  images  of  the  lower  altitude  sequences  are  matched 
directly  against  the  higher  altitude  ones.  For  both  methods  the  resulting  set  of  geometric 
constraints  is  used  in  a  bundle  adjustment  step,  which  promotes  the  final  geometric  accuracy. 
The  methods  were  evaluated  in  terms  of  accuracy  and  computational  efficiency.  We  found  the 
first  method  to  be  significantly  faster  while  marginally  less  precise. 

When  compared  to  the  case  of  single  altitude  sequences,  we  show  that  by  combining  geometric 
information  from  different  sequences,  we  are  able  to  successfully  estimate  the  topology  of  much 
lower  altitude  sequences.  This  allows  for  creating  benthic  views  with  higher  resolution  and 
geometric  accuracy. 

An  illustrative  result  is  presented  with  a  low  altitude  sequence,  captured  at  approx.  1.4  m  above 
sea  floor.  It  contains  benthic  structures  with  distinct  3D  content,  such  as  colonies  that  protrude 
50-70  cm  above  the  floor.  When  using  this  sequence  alone,  the  3D  content  and  the  low  overlap 
among  strips  leads  to  inaccurate  mapping  (Figure  A4  Left).  By  combining  with  a  second 
sequence  captured  at  approx.  2.5  m,  we  are  able  to  create  a  mosaic  of  much  higher  geometric 
accuracy  (Figure  A4  Right). 
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Figure  A4.  -  Left:  Mosaic  created  using  the  global  mosaicing  approach  on  a  low  altitude  sequence  alone.  The 
low  overlap  between  strips  leads  to  large  registration  errors  which  are  clearly  visible.  The  first  two  strips  on 
the  left  hand  side  should  be  overlapping.  Note  the  broken  image  of  the  quadrat,  made  of  white  PVC  tubing. 
Right:  The  improved  low  altitude  mosaic  resulting  created  using  geometric  information  from  both  the  low 
and  high  altitude  sequences.  Improvements  are  easily  visualized  when  comparing  the  shapes  of  the  4  PVC 
quadrats  in  the  two  examples. 


This  work  resulted  was  presented  orally  at  the  IEEE  conference  on  Ocean  Engineering  in 
Washington  DC  in  2005  and  was  published  in  the  following  journal: 

Gracias,  N.  and  Negahdaripour,  S.  (2005)  Underwater  Mosaic  Creation  Using  Video  Sequences 
from  Different  Altitudes.  Proc.  IEEE/MTS  Oceans’05,  Washington,  DC. 


A2.1.3.2  Efficient  Image  Blending 

Image  blending  is  an  integral  final  step  in  mosaic  processing  in  which  information  from  the 
hundreds  or  thousands  of  component  frames  used  in  the  image  matching  steps  are  combined  to 
create  a  single  large  image.  The  most  straightforward  method  for  assembling  a  single  mosaic 
from  multiple  registered  images  is  to  simply  fill  the  mosaic  with  the  portion  of  each  single  image 
closest  to  its  center.  This  approach  was  implemented  in  the  basic  mosaicing  package,  but  it  often 
left  visible  seams  in  the  mosaic  at  the  sudden  transitions  between  component  images.  These 
visible  seams  from  blending  are  distracting  to  the  user  and  can  interfere  with  interpretation  of  the 
mosaic  (Figure  A5). 
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Figure  A5.  Mosaic  rendering.  Left:  Area  detail  illustrating  a  mosaic  rendered  using  a  purely  geometric 
method  -  closest  distance  to  each  image  center.  Right:  Result  using  our  improved  blending  approach;  this 
method  uses  geometric  and  photometric  information,  to  compute  image  boundaries  that  avoid  cutting  benthic 
structures. 


During  2006  and  2007,  a  new  approach  was  devised  to  blend  images  using  watershed 
segmentation  and  graphcut  optimization  (Figure  A6.).  Watershed  segmentation  allows  for 
grouping  sets  of  neighboring  pixels  into  clusters  which  are  then  treated  as  indivisible  units  in  the 
blending  process.  This  leads  to  substantial  time  saving  since  the  number  of  clusters  is  much 
smaller  than  the  number  of  pixels.  Graphcut  optimization  is  a  recent  energy  minimization 
method  that  is  particularly  well  adapted  to  image  segmentation  applications.  Our  improved 
blending  method  presents  important  advantages  for  underwater  mosaics  over  existing  state-of- 
the-art  approaches.  It  combines  geometric  and  photometric  criteria  to  effectively  erase  seams 
between  images.  The  algorithm  stores  data  in  an  efficient  way,  thereby  reducing  memory 
requirements,  and  is  suited  to  parallel  implementation.  It  allows  the  efficient  blending  of  large 
mosaics,  with  no  user  intervention.  The  method,  referred  to  as  watershed/graph-cut  blending,  has 
been  presented  at  an  international  conference  (Gracias  et  al.  2006)  and  later  published  in  the 
following  journal  publication: 

Gracias,  N.,  M.  Mahoor,  S.  Negahdaripour,  and  A.  Gleason.  (2009)  Fast  Image  Blending  using 
Watersheds  and  Graph  Cuts,  Image  and  Vision  Computing,  27(5):  597-607. 
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Figure  A6.  Example  of  the  watershed/graph-cut  method.  The  region  of  intersection  of  the  two  images 
(marked  as  Rinter  in  (a))  is  divided  into  small  segments  using  watershed  segmentation  (b).  The  white  regions  in 
(b)  represent  the  areas  where  the  two  images  are  most  similar  in  terms  of  color  and  intensity.  These  are  the 
areas  where  the  seams  should  be  placed.  The  graph-cut  optimization  is  used  to  assign  each  segment  to  each 
image  in  order  to  reduce  the  overall  visibility  of  the  seam  (c).  The  resulting  blended  image  is  shown  in  (d). 


Two  additional  improvements  to  the  blending  procedures  have  been  incorporated  into  the  mosaic 
deliverable: 

i.  A  new  selection  scheme  to  reduce  the  number  of  images  to  be  blended.  The 
mosaicing  of  large  areas  may  involve  the  registration  and  joint  optimization  of  several 
thousand  images.  This  number  can  be  particularly  high  under  difficult  mosaicing 
conditions,  namely  in  the  presence  of  high  topography,  moving  algae  and  reduced 
visibility,  where  the  amount  of  overlap  between  images  needs  to  be  high  to  insure  proper 
registration.  Blending  many  images  with  high  overlap  often  leads  to  each  image  having 
a  small  contribution  on  the  final  mosaic  and  to  a  large  number  of  seams  among 
neighboring  images.  To  address  this  issue,  an  image  selection  scheme  was  devised  to 
find  a  much  smaller  sub-set  of  images  to  be  blended.  The  selection  insures  that  the  area 
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covered  by  the  mosaic  remains  the  same,  and  leads  to  substantial  time  savings  on  the 
subsequent  blending.  An  example  is  given  in  figure  A7. 


Figure  A7.  Example  of  frame  selection,  used  before  the  watershed/graph-cut  blending  method.  The  original 
set  of  691  images  (a)  is  reduced  to  145  images  (b).  The  frame  selection  method  rejects  images  with  small 
footprints  while  ensuring  enough  overlap  among  neighboring  images  required  by  the  watershed/graph-cut 
blending  method.  The  total  area  of  the  mosaic  is  not  affected. 

ii.  A  new  complementary  blending  method  using  image  gradients.  A  new  method  was 
developed  to  improve  the  final  rendering  of  the  mosaics.  It  uses  the  derivatives  of  the 
pixel  values  to  impose  smooth  transitions  between  neighboring  images.  This  method 
complements  the  existing  watershed/graph-cut  blending  module.  The  purpose  of  the 
watershed/graph-cut  blending  method  is  to  place  the  seams  (i.e.  the  common  border 
among  pairs  of  neighboring  images)  where  these  seams  are  the  least  visible.  This  means 
that  the  seams  are  placed  in  the  areas  where  the  neighboring  images  have  similar  pixel 
color  and  intensity.  However,  in  the  cases  where  the  neighboring  images  have  very 
different  intensities  (for  example  due  to  uncorrected  refracted  sunlight);  the 
watershed/graph-cut  method  is  not  able  to  totally  eliminate  the  visibility  of  the  seams. 
Conversely,  the  purpose  of  this  new  gradient  blending  module  is  to  smooth  the  transition 
of  pixel  color  and  intensity  over  the  pre-established  seam.  The  smoothing  is  done  by  (1) 
computing  the  horizontal  and  vertical  derivatives  of  the  mosaic,  (2)  imposing  zero 
derivatives  along  the  seams,  and  (3)  integrating  the  derivative  field  to  obtain  a  smoothed 
mosaic.  The  last  step  involves  performing  an  approximation,  since  the  derivatives 
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modified  in  (2)  cannot  be  directly  integrated.  This  approximation  is  done  by  solving  a 
Poisson  equation  using  Fourier  analysis.  An  example  of  the  result  of  the  new  gradient 
blending  module  is  given  in  Figure  A8. 


Figure  A8.  Comparison  of  blending  methods.  Fig  (a)  was  obtained  with  the  standard  blending  using  all 
images.  For  each  point  in  the  mosaic,  the  standard  method  selects  the  contribution  from  the  closest  image, 
which  results  in  a  large  number  of  visible  seams.  Fig  (b)  presents  the  result  of  the  watershed/graph-cut 
method  after  selecting  a  subset  of  representative  images  (see  text).  The  watershed/graph-cut  method  places 
the  seams  over  the  areas  where  they  are  the  least  visible.  However  seams  are  still  visible  between  images 
captured  under  different  illumination.  Fig  (c)  show  the  result  of  the  new  gradient  blending  module  applied  to 
the  result  of  (b).  The  image  transitions  are  smoothed  out,  thus  making  the  seams  practically  invisible. 

A2.1.3.3  Removal  of  Sunflickering  interference 
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A  common  problem  in  video  surveys  in  very  shallow  waters  is  the  presence  of  strong  light 
fluctuations,  due  to  sunlight  refraction.  Refracted  sunlight  casts  fast  moving  patterns,  which  can 
significantly  degrade  the  quality  of  the  acquired  data. 

A  new  module  for  the  removal  of  refracted  sunlight  from  shallow  water  video  was  created.  This 
module  is  based  on  a  method  developed  earlier  at  a  proof-of-concept  level. 

The  method  of  sunflickering  removal  exploits  the  fact  that  video  sequences  allow  several 
observations  of  the  same  area  of  the  sea  floor,  over  time.  The  image  difference  between  a  given 
reference  frame  and  the  temporal  median  of  a  registered  set  of  neighboring  images  is  computed 
and  used  to  correct  the  original  image  frame.  A  key  observation  is  that  this  difference  will  have 
two  components  with  separable  spectral  content.  One  is  related  to  the  illumination  field  (lower 
spatial  frequencies)  and  the  other  to  the  registration  error  (higher  frequencies).  The  illumination 
field,  recovered  by  lowpass  filtering,  is  used  to  correct  the  reference  image.  In  addition  to 
removing  the  sunflickering  patterns,  an  important  advantage  of  our  approach  is  the  ability  to 
preserve  the  sharpness  in  the  flicker-free  image,  even  in  the  presence  of  registration  inaccuracies. 

The  intermediate  results  of  the  method  are  illustrated  in  figure  A9.  This  example  was  constructed 
from  stack  of  nine  registered  frames  over  a  relatively  flat  area.  A  low  resolution  median  was 
constructed,  and  then  subtracted  from  the  original  image.  The  two  components  of  the  difference 
image  are  easily  seen.  They  can  be  effectively  separated  into  the  low  frequency  illumination  field 
and  the  high  frequency  residues  (resulting  from  imprecise  registration  and  the  low  resolution 
median). 
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Figure  A9.  Example  of  application  of  the  motion  compensated  filtering  algorithm  -Original  (reference)  frame 
(a),  sub-sampled  temporal  median  of  the  registered  stack  after  projection  into  the  reference  frame  (b),  image 
difference  between  the  median  image  and  the  original  for  one  of  the  color  channels(c),  low— pass  filtered 
version  of  the  difference  (d),  result  of  the  illumination  correction  to  the  original  image  (e),  and  difference 
between  the  corrected  image  and  the  median  (f).  The  values  of  images  (c),  (d)  and  (f)  have  been  stretched  to 
improve  visualization. 


The  method  was  tested  on  several  underwater  video  sequences.  Figure  A10  provides  an  example 
of  the  input  and  output  of  the  method  on  a  shallow  water  sequence  (2.5  meter  depth),  where  the 
camera  was  close  to  the  surface  and  moving  at  approximately  0.5  m/s.  The  method  is  able  to 
remove  the  sunlight  patterns  without  degrading  image  detail.  Figure  All  illustrates  the  impact  of 
refracted  sunlight  and  the  benefit  of  its  removal,  on  a  mosaic  section  of  a  shallow  water  survey. 
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Figure  A10.  Original  frames  of  a  shallow  water  sequence  (2-3  meter)  with  refracted  sunlight  (upper  row) 
and  resulting  frames  after  applying  our  filtering  technique  (lower  row). 


Figure  All.  Sections  of  the  same  shallow  water  survey  under  strong  refracted  sunlight,  using  the  original 
images  (a)  and  the  output  images  of  our  algorithm  for  sun-flickering  removal.  For  both  cases  the  same 
blending  method  was  used. 
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The  motivation  behind  the  sunflickering  work  was  mainly  practical,  due  to  the  large  number  of 
shallow  water  surveys  carried  out  on  sunny  days.  Unexpectedly  only  one  previous  work,  by 
Schechner  and  Karpel  (2004),  exists  on  the  topic  of  multi-image  sunflicker  removal.  Their 
method  is  based  on  the  median  of  gradients  and  is  inspired  by  effective  methods  for  removing 
shadows  in  open-air  scenes  (Weiss  2001).  This  approach  is  based  on  the  observation  that  the 
spatial  intensity  gradients  of  the  caustics  tend  to  be  sparse  ( i.e .  affect  small  areas  of  the  image). 
Therefore,  by  performing  the  temporal  median  over  the  gradients  of  a  small  number  of  images,  it 
is  possible  to  obtain  a  gradient  field  where  the  effects  of  the  varying  illumination  are  eliminated 
or  at  least  greatly  reduced.  The  flicker-free  image  is  obtained  by  integrating  the  median  gradient 
field.  As  the  gradient  field  is  likely  to  be  inconsistent,  a  least  squares  approximation  is  performed 
using  reversed  derivative  kernels.  This  approach  does  not  attempt  to  estimate  any  particular  time 
instance  of  the  refracted  sunlight  illumination  field.  Rather,  the  flicker-free  image  is 
reconstructed  using  contributions  from  all  images,  as  determined  by  the  temporal  median. 
Therefore  this  method  can  lead  to  blurry  results  under  camera  motion,  since  small  registration 
errors  are  practically  unavoidable. 

In  order  to  compare  performance  we  implemented  the  median  of  gradients  method,  closely 
following  the  algorithm  provided  in  (Schechner  and  Karpel  2004).  Figure  A12  provides  an 
example  using  5  registered  images  where  the  3D  content  of  the  scene  is  clearly  not  well 
represented  by  planar  image  motion.  The  method  based  on  the  median  of  gradients,  lead  to 
blurry  results  due  to  the  local  misalignments  from  the  3D  content,  whereas  our  method  is  able  to 
produce  a  sharp  flicker-free  image. 
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(b)  (c) 


Figure  A12.  Performance  of  sunflicker  removal  methods  under  poor  registration  —  Original  frame  from  a 
sequence  of  5  images  registered  under  the  assumption  of  flat  seafloor  (a),  result  of  the  median  of  gradients 
method  (Schechner  and  Karpel  2004)  showing  blurring  and  color  bleeding  (b),  and  result  of  our  method 
which  is  capable  of  removing  the  illumination  artifacts  without  degrading  the  image  quality  (c). 


Our  work  on  sunflickering  resulted  in  a  research  paper  (Gracias  et  al.,  2008)  that  was  submitted, 
accepted  and  presented  orally  at  the  IEEE  conference  on  Ocean  Engineering  in  Quebec  City, 
Canada  in  September  2008.  Details  on  the  method  and  a  formal  description  of  the  approach  are 
given  in  the  paper.  An  extended  version  is  in  preparation  for  journal  submission. 

The  initial  research  code  (used  for  producing  the  paper  results)  was  streamlined  into  an  easy-to- 
use  software  module  that  is  part  of  the  project  deliverable.  The  streamlining  was  done  at  two 
levels:  (1)  several  input  parameters  now  have  data-driven  values  that  are  automatically  adjusted 
from  analysis  of  the  images,  thus  releasing  the  user  from  that  requirement,  and  (2)  a  graphical 
user  interface  was  created  for  the  complete  process. 

A2.1.3.4  Profiling  and  identification  of  bottlenecks 

As  part  of  improving  the  overall  efficiency  of  mosaic  creation,  a  profiling  of  processing 
bottlenecks  was  conducted.  The  purpose  was  to  evaluate  the  entire  mosaic  production  pipeline 
and  identify  the  areas  that  need  improvement.  This  effort  lead  to  improvements  conducted  in 
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2008  and  2009,  which  involved  recoding  some  parts  of  the  software  with  gains  at  two  levels  - 
execution  speed  and  memory  limits. 

The  execution  speed  and  memory  limitations  were  addressed  in  two  ways,  and  are  detailed  in 
sections  A5.1.3.5  and  A5.1.3.6.  The  following  tables  (Table  A1  and  Table  )  summarize  the 
processing  steps  and  indicate  the  level  of  improvement  expected  after  re-coding.  The  first  table 
refers  to  the  existing  mosaic  creation  modules  and  the  second  to  the  newer  added  capabilities. 


Table  Al.  Table  showing  areas  where  speed  and  memory  efficiency  could  be  improved  in  existing  mosaic 
creation  modules. 


Task 

Subtask 

Time  presently 
required,  and  % 
of  total  (small 
size  mosaic) 

Suitability  for 
Parallel 

Implementation 

Impact  of 

Parallel 

Implem. 

Convertible 
to  C  /  Mex 

Impact  of 
conversion 
to  C  /  Mex 

Image 

extraction 

Image  extraction 

660  sec  (1.8%) 

Low 

Low 

No 

Very  low 

Global 

Matching 

Computing 

SIFTs 

608  sec  (1.6%) 

High 

Medium 

No 

Low 

Global 

Matching 

Image  Matching 

34600  sec  (95%) 

High 

Very  High 

Low 

Low 

Global 

Matching 

Optimization 

70  sec  (0.2%) 

None 

None 

No 

None 

Blending 

Image  mapping 

80  sec  (0.2%) 

Low 

Low 

Yes 

Medium 

Blending 

Image  Warping 

300  sec  (0.8%) 

Low 

Low 

No 

None 

Table  A2.  Table  showing  areas  where  of  speed  and  memory  efficiency  could  be  improved  in  enhanced  mosaic 
capabilities. 


Added 

Capability 

Suitability  for 
Parallel 

Implementation 

Impact  of 

Parallel 

Implem. 

Convertible 
to  C  /  Mex 

Impact  of 
conversion 
to  C  /  Mex 

Reduction  of 
Sun  flickering 

Low 

Low 

Partially 

Low 

Use  of  Heading 
Sensor 

None 

None 

No 

Low 

Graphcut 

Blending 

High 

Low 

Partially 

Medium 

Integration  of 
Stills  and 

Video 

High 

High 

No 

None 

The  following  routines  were  identified  as  key  candidates  that  would  benefit  from  recoding: 

Image  mapping  in  Blending,  and  Graphcut  Blending  -  These  steps  contain  very  repetitive  low- 
level  operations  which  can  be  suitably  implemented  in  C/C++. 

Computing  SIFTs,  Image  matching,  and  Integration  of  video  and  stills  -  A  substantial  part  of 
these  routines  operate  on  pairs  of  images.  The  processing  of  each  pair  is  totally  independent  of 
any  other  pair.  Thus  there  is  a  potential  for  large  speed  increase  after  parallelization. 
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The  memory  limitations  affect  mostly  the  global  optimization  and  the  blending  part  of  the 
mosaic  creation.  The  limitations  were  imposed  by  the  Matlab  version  6.5,  now  outdated,  for 
which  the  code  was  developed. 


A2.1.3.5  Memory  Enhancements 


Memory  limitations  of  the  original  mosaic  deliverable  product  stemmed  from  problems  inherent 
in  running  the  mosaicing  algorithm  in  an  outdated  version  of  Matlab.  Although  version  6.5  of 
Matlab  was  the  programming  environment  in  which  the  mosaicing  processing  algorithms  were 
created,  this  version  is  bound  to  32bits  architectures  with  a  maximum  size  for  a  single  data 
structure  of  1GByte.  As  an  example,  this  environment  precludes  the  rendering  of  mosaics  larger 
than  50  Mpixel  thereby  limiting  the  size  and  resolution  of  all  mosaic  products  produced  in  this 
format.  Conversion  and  adaptation  of  the  code  to  the  newer  version  of  Matlab  and  to  a  64  bit 
version  of  the  operation  system  raised  this  limit  considerably. 

All  components  of  the  mosaic  creation  code  were  successfully  converted  from  Matlab  version 
6.5  (released  in  2002)  to  version  2007b/64  bit  (released  in  late  2007).  Among  other  advantages, 
the  newer  64  bit  version  allows  for  significantly  larger  data  structures,  thus  releasing  the  existing 
memory  limitation.  With  this  conversion,  the  processing  is  only  limited  by  the  amount  of 
physical  memory  in  the  computer.  In  a  PC  with  8GB  RAM,  the  mosaic  software  in  now  able  to 
produce  mosaics  in  excess  of  300  Mpixel,  which  are  six  times  larger  than  previously  possible. 
This  allows  us  to  not  only  process  larger  datasets  but  to  incorporate  higher  resolution  imagery 
into  our  mosaic  processing  systems.  Both  changes  have  resulted  in  improved  products  for  coral 
reef  monitoring. 

Apart  from  the  conversion  to  the  64  bits  platform,  several  improvements  were  made  on  the  data 
structures  to  promote  a  lower  use  of  memory.  Among  these  was  a  new  tiling  scheme  for  the 
gradient  blending  method  The  new  gradient  blending  module  requires  the  use  of  the  2D  Fast 
Fourier  Transform  algorithm  (FFT)  and  its  inverse  (IFFT).  The  FFT  and  IFFT  are  applied  to 
matrices  that  are  as  large  as  the  final  mosaic  size.  For  very  large  mosaics  (of  more  than  hundred 
Mpixels)  this  requires  enormous  amounts  of  memory  which  are  not  available  in  standard  or  high- 
end  PCs.  In  order  to  address  this  issue,  we  have  created  a  scheme  to  divide  the  mosaic  space  into 
smaller  tiles.  Each  tile  is  processed  individually,  taking  into  account  the  tile  border  conditions  (to 
avoid  creating  seams  along  these  borders).  This  scheme  allow  for  running  the  gradient  blending 
method  for  very  large  mosaics  in  PCs  with  standard  memory  capacity  (3GByte).  Furthermore  the 
processing  of  all  the  tiles  is  significantly  faster  than  a  single  run  on  the  whole  mosaic. 


A2.1.3. 6  Speed  Enhancements 
Parallelization 

A  central  limitation  of  our  previous  mosaic  processing  capability  was  the  single  thread 
processing  of  many  key  routines.  As  such  we  have  recoded  many  of  these  routines  into  C 
language,  using  multithreaded  programming.  Multithread  programming  allows  for  low-level 
parallelization  in  processors  with  several  cores.  For  example  the  lower-level  blending  routines 
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operate  at  pixel-level,  and  perform  a  very  large  number  of  independent  operations  which  can  be 
executed  in  parallel.  The  recoding  into  multi-threaded  C  language  resulted  in  an  eight-fold 
increase  of  speed  in  our  dual  quad  core  PC. 

Improvement  in  the  linear  optimization  module  -  Implementation  of  a  rigid  sub-mosaic  approach 
A  central  component  in  the  mosaic  creation  process  is  the  so-called  global  optimization  where 
the  relative  displacements  of  all  images  are  computed.  This  process  is  carried  out  in  two  stages: 
(1)  linear  optimization  to  provide  a  fast,  approximate  solution  and  (2)  non-linear  optimization  to 
improve  on  that  solution.  The  linear  optimization  module  requires  the  inversion  of  very  large 
matrices,  and  sets  the  limit  on  the  number  of  images  that  can  be  simultaneously  mosaiced.  Due  to 
the  memory  limitations  of  version  6.5  of  Matlab,  the  matrix  inversion  could  not  be  performed  for 
mosaics  of  more  than  2000  images.  This  became  a  limiting  factor  for  mosaicing  larger  areas.  The 
conversion  to  Matlab  version  2007b/64bit  allowed  for  inverting  matrices  of  much  larger  sizes. 
However  the  execution  time  for  matrix  inversion  grows  cubically  with  the  size  of  the  matrix. 
When  doubling  the  number  of  images  the  processing  time  increases  8-fold. 

To  address  this  issue,  a  new  linear  optimization  module  was  created  in  which  the  number  of 
variables  is  reduced  by  considering  several  small  rigid  sub-mosaics.  Using  sub-mosaics  of  5  to 
10  images  each,  the  linear  solution  is  125  to  1000  times  faster,  without  adverse  effects  to  the 
non-linear  optimization  stage.  With  the  new  module,  more  than  7000  images  can  be  processed 
efficiently. 

Improvement  in  the  non-linear  optimization  module  -  Computation  of  analytic  derivatives 
instead  of  numerical  approximations 

The  non-linear  optimization  stage  searches  for  the  best  displacement  of  images  in  the  mosaic,  by 
minimizing  a  given  cost  function.  It  requires  the  repeated  computation  of  the  derivatives  of  the 
cost  function  with  respect  to  the  parameters  being  minimized.  On  our  initial  implementation,  this 
computation  took  most  of  the  processing  time,  since  it  was  done  by  numerical  approximations. 
The  non-linear  optimization  module  was  re-coded  to  use  sparse  matrix  structures  and  to  compute 
the  derivative  from  symbolic  formulas,  instead  numerical  approximations.  As  a  result,  the 
execution  time  was  cut  by  a  factor  of  3  for  the  typical  number  of  images  in  our  mosaics  (about 
800  to  1200).  For  larger  sets,  the  speed  increase  is  even  larger. 

A2.1.4  Second-generation  landscape  mosaics 

The  second-generation  system  was  designed  and  built  to  incorporate  high  resolution  still  imagery 
and  real  time  heading  information  into  a  standardized  video  acquisition  system  that  would  be 
easily  deployed  and  utilized  by  a  single  research  diver.  The  purpose  of  this  upgraded  system  was 
to  increase  benthic  resolution  of  the  standard  2D  video  mosaic  products  (as  developed  during  this 
project)  and  incorporate  real  world  heading  information  into  the  video  acquisition  process  to 
assist  mosaic  processing  for  high  relief  reef  areas  (See  Section  A2. 1.4.2). 

The  upgraded  mosaicing  system  was  created  using  commercially  available  imaging  components 
including:  a  Nikon  D200  10MP  Digital  SLR  camera  and  Ikelite  underwater  housing,  Sony  HDV 
video  camera  and  Amphibico  housing,  fluxgate  compass,  and  3.5”  LCD  external  color  video 
monitor  (Figure  A13).  All  components  of  the  enhanced  mosaicing  system  were  mounted  in  an 
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aluminum/stainless  steel  frame. 


Fluxgate  compass,  Ikelite 
housing,  and  monitor 


Sony  HDV  camera  and  Amphibico 
housing 


3.5  “  LCD  color  monitor 


Nikon  D200  10MP  still  camera  and 
housing 


Figure  A13.  Completed  enhanced  imaging  system. 


Synchronization  of  the  digital  still  frames  with  the  video  was  achieved  by  installing  an  audio 
microphone  inside  the  still  digital  camera  housing  to  detect  the  click  noise  produced  when  the 
camera  shutter  is  activated  when  a  still  picture  is  taken.  This  audio  signal  is  routed  to  one  of  the 
audio  channels  of  the  video  camera  using  a  bulkhead  waterproof  connector. 

To  integrate  heading  information,  a  compass  interface  electronic  circuit  was  designed,  tested  and 
calibrated.  The  heading  information  is  provided  by  a  KVH  fluxgate  electronic  compass  that 
measures  the  Earth’s  magnetic  field  and  produces  an  analog  voltage  output  proportional  to  the 
compass  heading.  This  voltage  is  converted  to  a  frequency  in  the  audio  range  and  stored  in  the 
audio  channel  #1  of  the  high  definition  video  camera.  The  transfer  function  of  the  electronic 
circuit  was  measured  on  the  bench  obtaining  the  values  shown  in  Table  A3  and  plotted  as 
Figures  A14and  Figure  A15. 


Table  A3.  Measured  Transfer  Function 
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Degrees  Vin  (Volts)  Fout  (Hertz) 


0  0.1  236 

21.4  0.207  485 

43.2  0.316  737 

60.6  0.403  940 

81.4  0.507  1185 

101  0.605  1410 

123  0.715  1660 

140  0.8  1860 

163.6  0.918  2140 

193.4  1.067  2480 

224  1.22  2840 

241.6  1.308  3040 

260.6  1.403  3260 

281.2  1.506  3500 

300.4  1.602  3730 

323  1.715  4000 

341.2  1.806  4200 

359  1.895  4400 


Voltage  Transfer  Function 


0  0.2  0.4  0.6  0.8  1  1.2  1.4  1.6  1.0  2 

Volts  In 


Figure  A14.  Plot  of  the  Measured  Voltage  Transfer  Function 
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To  calibrate  the  compass,  the  system  was  mounted  on  a  rotating  table  with  the  real  time  LCD 
compass  display  mounted  in  front  of  the  cameras  enabling  the  video  camera  to  record  the 
displayed  compass  heading  while  recording  in  one  of  its  audio  channels  the  tone  corresponding 
to  this  particular  heading  and  also  recording  in  the  second  audio  channel  the  instant  that  a  still 
picture  was  taken  with  the  Nikon  camera.  The  system  was  rotated  following  a  pre-established 
protocol  while  the  video  camera  was  recording  the  heading  and  a  still  was  taken  every  two 
seconds.  Figure  A16  illustrates  the  calibration  setup. 


Figure  A16.  Heading  Calibration  Setup 
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The  upgraded  mosaic  imaging  system  was  designed  for  deployment  by  a  single  research  diver, 
and  is  neutrally  buoyant  in  saltwater.  The  imaging  system  is  designed  to  record  up  to  an  hour  of 
video  per  tape  and  up  to  1300  10MP  still  images  before  downloading.  The  interval  timer  mode 
of  the  Nikon  D200  allows  users  to  change  the  rate  of  image  acquisition  depending  on  survey 
requirements.  Field  testing  of  the  enhanced  mosaic  imaging  system  has  shown  that  surveys 
performed  with  still  images  taken  once  every  2  seconds  provided  sufficient  overlap  to  cover  a 
1  Ox  10m  area. 

A2A. 4.1  Integrating  high  resolution  stills  with  video  image  acquisition 

The  objective  of  this  effort  was  to  increase  the  resolution  of  the  mosaics  by  combining  images 
from  two  coupled  optic  sensors  (i.e.  cameras)  with  different  resolutions  and  frame-capture  rates. 
The  mosaic  algorithm  requires  a  high  overlap  between  sequential  images  to  accurately  estimate 
camera  motion.  In  the  field,  high  overlap  between  images  is  achieved  by  using  video  cameras 
with  a  high  rate  of  frame  capture.  In  addition,  the  ability  to  detect  and  identify  small  organisms 
(<  10  cm)  from  the  mosaic  requires  a  high  image  resolution.  Current  camera  technology  does  not 
provide  a  system  that  can  simultaneously  capture  frames  at  a  high  rate  with  high  image 
resolution. 

The  proposed  solution  was  to  combine  information  from  two  cameras:  (1)  a  high-definition 
video  camera  capable  of  high  frame-capture  rates  (30  fps)  with  lower  image  resolution  (< 
IMpixel);  and  (2)  a  digital  still  camera  capable  of  acquiring  very  high-resolution  images  (>  10 
Mpixel)  at  low  capture  rates  (one  image  every  sec). 

The  enhanced  acquisition  hardware  allows  for  the  simultaneously  collecting  video  and 
photographic  stills  using  two  independent  cameras  mounted  on  the  same  frame.  The  video  sensor 
can  provide  images  of  0.7  Mpixel  at  30  frames  per  second,  while  the  still  camera  provides  10 
Mpixel  images  at  1  frame  per  second.  The  sensors  are  not  synchronized,  so  the  correspondence 
between  the  video  and  the  still  frames  is  not  known  during  acquisition.  However,  the  sound  of 
the  shutter  release  of  the  still  camera  is  recorded  in  one  of  the  audio  channels  of  the  video 
camera.  The  approach  for  combining  video  and  stills  comprises  the  following  three  steps: 

1  -  Find  correspondences  between  individual  video  frames  and  stills. 

2  -  Create  a  mosaic  from  the  video  images. 

3  -  Relate  the  stills  with  the  mosaic. 

Step  1  deals  with  determining  which  video  frame  most  closely  relates  to  each  photo  still.  Two 
approaches  were  tested: 

1  -  Detecting  the  audio  signatures  of  the  shutter  release. 

2  -  Using  image  matching  between  video  and  stills. 

The  first  method  requires  the  user  to  establish  the  first  corresponding  pair  of  video  and  still.  It 
assumes  that  all  subsequent  shutter  releases  are  successfully  detected.  The  second  method  uses 
the  image  matching  tools  already  developed  for  the  mosaic  creation.  It  does  not  require  user 
intervention,  is  significantly  more  robust  in  finding  the  correct  correspondences,  and  provides  the 
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geometric  displacement  between  images.  Albeit  slower  (taking  a  few  hours  instead  of  few 
minutes),  it  was  the  method  of  choice. 

Step  2  is  the  standard  mosaic  creation  procedure.  In  a  final  step  the  information  of  the  first  two 
steps  is  combined  to  establish  the  position  of  each  still  in  the  mosaic  frame. 

Results  of  the  joint  processing  of  video  images  and  photo  stills  were  obtained  using  data 
collected  in  Andros,  Bahamas  in  October  2007.  Figure  A17  illustrates  the  results  of  matching 
pairs  of  images  from  the  two  different  sensors.  For  each  video  frame,  this  matching  allowed 
establishing  the  closest  corresponding  still,  by  choosing  the  largest  overlap  between  frames. 
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Figure  A17.  Table  representing  matches  between  video  images  and  photo  stills.  Successful  matches  are 
represented  by  dark  dots. 

A  simple  point  and  click  tool  was  created  to  illustrate  the  usefulness  of  combining  video  and 
stills.  The  tool  allows  for  retrieving  the  video  image  and  the  still  photo  that  are  closest  to  a  point 
of  interest  in  the  mosaic,  defined  by  the  user.  Figure  A17  illustrates  the  concept. 

This  tool  was  further  developed  into  a  stand-alone  Windows  application.  It  provides  the  same 
point  and  click  functionality  to  retrieve  and  inspect  individual  video  and  still  images.  Since  it 
does  not  require  the  presence  of  the  Matlab  environment  in  the  computer.  It  can  easily  be 
distributed  and  used  in  any  windows  machine.  The  application  is  described  in  Section  A2.6.3 

A2.1.4.2  Integration  of  heading  information  in  2D  mosaic  processing 

In  a  further  attempt  to  improve  mosaicing  in  areas  of  high  topographic  relief,  we  investigated 
integration  of  heading  information  to  help  track  the  trajectory  of  the  video  cameras.  The 
detection  of  camera  trajectory  is  a  crucial  step  needed  to  obtain  high  quality  mosaics  with  low 
geometric  distortions.  The  proposed  solution  was  to  integrate  a  heading  sensor  with  the  video 
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camera,  in  order  to  provide  absolute  orientation  of  each  image  frame  with  respect  to  North.  The 
heading  sensor  allows  all  frames  to  be  oriented,  thus  increasing  the  chances  of  tracking  the 
correct  camera  trajectory. 

Integration  of  a  heading  sensor  for  improved  mosaicing  in  high  relief  areas  involved  two  steps: 
i  -  Develop  algorithms  to  decode  the  compass  output  information.  This  information  is  recorded 
in  one  of  the  audio  channels  of  the  High  Definition  Video  (HDV)  camera. 

ii-  Develop  a  calibration  procedure  to  obtain  a  relationship  between  the  output  of  the  compass 
and  the  heading  values  and  to  assess  the  sensor  accuracy. 

i.  Decoding  the  heading 

The  flux  gate  compass  outputs  a  voltage  proportional  to  the  heading  angle.  This  voltage  is 
converted  into  a  squared  wave  of  varying  frequency  using  a  Voltage  Controlled  Oscillator 
(VCO).  The  frequency  ranges  from  250Hz  to  4400Hz,  approximately  corresponding  to  heading 
angles  of  0  to  360  degrees.  The  camera  is  set  to  record  at  48  KHz  thus  being  able  to  reproduce 
signals  of  frequencies  up  to  its  Nyquist  cut-off  frequency  of  24  KHz.  Thus  the  frequencies  of  the 
heading  signal  are  well  within  the  range  adequately  recorded  in  the  audio  channel  of  the  camera 
without  aliasing. 

Each  cycle  of  a  squared  wave  comprises  a  high  value  output  and  a  low  value  output.  The  duty 
cycle  is  defined  as  the  ratio  of  the  duration  of  the  high  value  output  with  the  duration  of  the 
complete  cycle.  The  squared  wave  produced  by  the  VCO  has  a  duty  cycle  that  varies  with  the 
frequency.  Figure  A18  illustrates  audio  samples  for  a  low  frequency  and  a  high  frequency  case. 


Audio  Sample  (window  index  5510) 


Time(samples  at  48Khz) 
Audio  Sample  (window  index  2340) 


Time(samples  at  48Khz) 


Figure  A18.  Examples  of  audio  samples  for  a  low  frequency  case  (top,  250Hz  corresponding  to  heading  of 
approximately  0  deg)  and  high  frequency  case  (bottom,  4400Hz  corresponding  to  approximately  360  deg). 

Our  goal  was  to  determine  the  base  frequency  of  the  square  wave  (first  harmonic).  The  fact  that 
the  duty  cycle  is  not  constant  results  in  distinct  spectral  content  for  wave  of  different  base 
frequencies.  To  determine  the  base  frequency,  the  audio  signal  is  divided  into  non-overlapping 
segments  of  1600  samples.  Each  segment  has  the  duration  of  one  frame  of  the  original  HD 
video,  at  30  frames  per  second  (fps),  and  is  processed  independently. 
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To  analyze  the  frequency  content  of  each  segment,  three  approaches  were  considered.  These  are 
based  on  computing  the  values  of:  1  -  the  power  spectral  density,  2  -  the  temporal 
autocorrelation  and  3  -  the  magnitude  of  the  Fast  Fourier  Transform.  A  representation  of  these 
computations  is  given  in  Figure  A19  for  low  frequency  and  high  frequency  examples. 


FFT  Coeff  Idx  (samples  at  48Khz)  FFT  Coeff  Idx  (samples  at  48Khz) 

Figure  A19.  Examples  of  audio  samples  (first  row),  and  corresponding  power  spectral  density  (second  row), 
temporal  autocorrelation  (third  row)  and  magnitude  of  the  FFT  (forth  row),  for  the  low  frequency  case  (left 
column)  and  the  high  frequency  case  (right  column). 


All  three  approaches  can  be  used  to  determine  the  base  frequency,  by  localizing  the  first  peak. 
However  the  best  results  were  obtained  by  the  magnitude  of  the  FFT.  This  approach  is  the  most 
robust  to  artifacts  caused  by  changes  of  base  frequency  within  the  segment.  Changes  in 
frequency  occur  when  the  sensor  outputs  a  new  heading  value. 

An  unexpected  issue  occurs  when  the  heading  sensor  crosses  the  0  to  360  degree  boundary.  The 
heading  sensor  has  relatively  large  settling  time  and  thus  outputs  intermediate  values  over  an 
interval  of  approximately  one  second.  These  intermediate  values  need  to  be  filtered  out,  since 
they  will  lead  to  largely  inconsistent  heading  readings.  In  addition,  the  sensor  has  gaps  in  the 
output  when  it  changes  its  output  values.  These  gaps  last  for  approximately  50  msec,  during 
which  the  frequency  is  close  to  zero,  and  can  corrupt  the  segments  to  the  point  of  resulting  in 
wrong  base  frequency  estimation. 

The  issues  of  the  large  settling  times  and  the  output  gaps  were  solved  by  (1)  performing  a 
filtering  of  the  estimated  base  frequency  using  the  median  over  a  temporal  sliding  window,  and 
(2)  explicitly  detecting  the  north  crossing  transitions  and  removing  intermediate  outputs  during 
the  settling  time.  An  illustration  of  this  process  is  given  in  Figure  A20. 
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Audio  Values  for  Transition  at  North  (360  to  0  deg) 
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Figure  A20.  Illustration  of  the  large  settling  time  for  a  360  to  0  degree  transition.  The  top  figure  contains  an 
audio  sample  for  an  interval  of  approximately  one  second.  The  lower  left  figure  shows  the  raw  value  of  the 
decoded  heading  signal,  which  is  proportional  to  the  audio  frequency.  The  gaps  and  the  multiple  intermediate 
frequencies  are  visible  in  the  several  peaks.  The  lower  right  figure  shows  the  raw  decoded  heading  after 
filtering,  with  these  artifacts  removed. 


ii.  Heading  sensor  calibration 

The  purpose  of  the  sensor  calibration  is  twofold:  (1)  to  obtain  a  way  to  convert  the  frequency 
output  values  into  heading  angles,  and  (2)  check  for  systematic  errors  caused  by  the  magnetic 
interference  of  the  hardware  surrounding  the  compass,  and  estimate  sensor  accuracy. 

A  calibration  procedure  was  devised.  It  comprises  the  following  steps: 

1 .  Acquire  video  under  pure  vertical  rotation  -  The  acquisition  is  done  with  the  complete 
setup,  in  the  water,  with  the  camera  roughly  pointing  towards  the  same  area  in  the 
bottom. 

2.  Use  image  registration  to  obtain  rotation  angles  between  images  -  This  involves 
performing  the  global  matching  step  of  the  2D  mosaic  creation  software.  Since  all  images 
look  at  the  same  area  of  the  bottom,  it  is  possible  to  perform  the  image  registration 
between  any  pair  of  images.  Therefore,  there  we  can  estimate  the  rotation  angle  of  each 
image  with  respect  to  the  first  image  (considered  the  reference  frame)  without 
accumulating  errors.  This  measurement  is  taken  as  ground-truth  and  compared  against  the 
compass  measurements. 

3.  Fit  a  curve  to  the  pairs  of  compass  reading  and  image  rotation  angles. 

4.  Characterize  distribution  of  residues,  to  check  for  systematic  errors  and  obtain  a  sensor 
accuracy  estimate. 

The  image  rotation  angles  measured  from  the  calibration  sequence  and  the  sensor  readings  are 
plotted  in  Figure  A21 . 
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Figure  A21.  Rotation  angles  obtained  from  the  calibration  sequence  using  image  registration  (upper  row) 
and  raw  sensor  readings  decoded  from  the  compass  audio  recording  (lower  row). 


Figure  A22  (left)  shows  that  the  pairs  of  compass  readings  and  image  angle  measurements  have 
a  close  linear  relationship.  Additionally,  no  large  deviation  from  this  line  is  observed,  which 
indicates  the  absence  of  large  systematic  errors.  As  such,  there  was  no  need  to  correct  for  these 
errors. 

The  uncertainty  of  the  heading  sensor  was  estimated  by  checking  the  distribution  of  residues  of 
the  linear  fit.  The  histogram  is  presented  in  Figure  A22  (right),  and  approximately  follows  a 
Normal  distribution  with  standard  deviation  of  2.2  degrees.  This  value  is  consistent  with  the 
rated  precision  reported  by  the  manufacturer,  and  is  well  suited  for  our  needs. 
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Image  angles  vs  Sensor  output 


Degrees 


Figure  A22.  Calibration  of  the  heading  sensor  -  The  left  figure  shows  pairs  of  compass  readings  and  image 
angle  measurements  (blue  dots)  with  a  linear  fit  (red  line).  The  right  figure  presents  the  histogram  of 
residues  from  the  linear  fit.  The  red  line  is  a  Normal  distribution  with  the  same  zero  mean  and  the  same 
standard  deviation  of  2.24  degrees  as  the  residues. 


To  investigate  the  usefulness  of  the  heading  information,  we  performed  two  experiments.  The 
experiments  differ  regarding  the  stage  of  the  mosaic  creation  process  where  the  heading 
information  was  used. 

The  current  implementation  of  the  2D  mosaicing  algorithm  assumes  that  a  set  of  images  have 
been  extracted  from  a  video  sequence.  In  the  first  step,  called  initial  trajectory  estimation,  the 
images  are  matched  consecutively  in  pairs  along  time.  The  purpose  of  this  step  is  to  remove 
images  that  have  high  overlap  with  the  previous  ones,  thus  reducing  the  number  of  total  images 
to  be  mosaiced.  In  the  second  step,  called  global  matching,  the  selected  images  are  matched 
against  the  others  in  order  to  establish  as  many  links  as  possible.  These  links  are  geometric 
constraints  among  images  that  result  from  a  successful  image  matching.  In  the  third  step,  called 
global  alignment,  all  the  image  links  are  taken  into  account  to  establish  the  best  geometric 
arrangement  for  all  images. 

The  first  result  is  an  a  posteriori  application  of  the  heading  information  to  a  globally  aligned 
mosaic  that  was  created  using  information  from  the  image  matching  alone.  We  selected  an 
example  where  the  presence  of  high  topography  prevented  the  mosaicing  algorithm  from 
producing  geometrically  accurate  results  (Figure  A23).  The  survey  pattern  consists  of  horizontal 
strips.  The  two  central  strips  have  very  low  overlap.  A  close  inspection  of  the  right  hand  side  of 
the  mosaic  reveals  that  the  mosaic  section  above  and  below  the  wedge-shaped  dent  should  be 
slightly  overlapping.  Due  to  the  low  superposition  of  these  images  and  high  topography  area  in 
the  middle,  the  mosaic  algorithm  did  not  join  these  sections. 
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Figure  A23.  Example  of  a  globally  aligned  mosaic  with  geometric  distortions.  The  gap  inside  the  area 
marked  in  yellow  should  not  exist  since  the  upper  and  lower  portions  of  mosaic  overlap.  The  arrows  mark 
the  same  benthic  structures.  The  distortions  are  due  to  high  relief  (in  the  central  area  of  the  mosaic)  and  low 
overlap  among  the  images  inside  the  marked  region. 

The  heading  information  is  used  by  replacing  the  orientation  of  the  images  usually  obtained  by 
the  global  alignment,  with  the  compass  readings.  The  relative  translation  among  neighboring 
images  along  the  trajectory  was  kept  intact.  This  process  changes  both  the  position  and  the 
orientation  of  each  image,  and  has  the  effect  of  restoring  the  parallelism  among  the  horizontal 
strips  that  form  the  survey  pattern. 

A  schematic  of  the  image  displacement  before  and  after  the  use  of  heading  is  given  in  Figure 
A24.  The  use  of  heading  leads  to  a  small  improvement,  since  the  two  sides  of  the  gap  are  closer. 
However  it  is  not  sufficient  to  remove  the  gap  and  join  the  overlapping  areas. 
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Figure  A24.  Example  of  the  use  of  heading  on  a  globally  aligned  mosaic.  The  mosaic  on  the  left  was  obtained 
from  image  matching  alone  and  is  the  same  in  Figure  A23.  The  mosaic  on  the  right  was  created  by 
combining  the  heading  information  from  the  compass  with  the  image  translation  provided  by  the  image 
matching.  The  heading  improved  the  results  but  not  enough  to  avoid  the  gap  on  the  right.  These  mosaics 
were  rendered  with  visible  image  frames  to  emphasize  differences  in  orientation. 


The  second  example  illustrates  a  potential  benefit  of  using  heading  during  the  step  of  initial 
trajectory  estimation.  Figure  A25  shows  an  example  from  a  short  video  sequence  where  the 
camera  moved  sideways  while  keeping  roughly  the  same  heading.  The  survey  pattern  consisted 
of  4  parallel  strips.  Without  heading  information  the  initial  trajectory  rapidly  accumulates  errors 
(namely  in  the  orientation  of  the  images),  which  leads  to  large  displacements  in  the  trajectory. 
Using  the  heading  information  the  error  growth  is  much  smaller,  resulting  in  a  trajectory  closer 
to  what  is  obtained  after  the  complete  mosaicing  process. 

As  mentioned  previously,  in  the  current  implementation  of  the  mosaicing  algorithm,  the  initial 
trajectory  estimation  step  is  only  used  to  check  the  superposition  among  time-consecutive 
images,  and  to  discard  redundant  images.  However,  if  the  trajectory  is  accurate  enough,  it  can  be 
used  to  predict  areas  of  possible  image  overlap.  The  global  matching  step  (that  follows)  can 
concentrate  on  attempting  to  match  images  in  these  areas,  where  successful  matches  are  more 
likely  to  be  found.  This  will  potentially  have  a  large  impact  in  reducing  the  amount  of 
processing  time  required  for  the  global  matching  step. 
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(A)  (C) 


Figure  A25.  Example  of  use  of  heading  in  the  estimation  of  the  initial  trajectory.  In  this  short  video,  the 
camera  moved  sideways,  keeping  roughly  the  same  heading.  The  survey  pattern  consists  of  4  strips,  which 
are  approximately  parallel.  (A)  is  the  initial  trajectory  estimate  without  heading  information.  The  motion 
estimation  errors  accumulate  rapidly  resulting  in  non-parallel  strips.  (B)  is  the  initial  trajectory  where  the 
heading  was  obtained  from  the  compass  and  the  translation  from  the  image  matching.  The  error 
accumulation  is  smaller  since  the  strips  remain  parallel.  (C)  is  the  result  of  the  complete  mosaicing  process 
using  global  matching  and  global  estimation,  and  serves  here  as  ground-truth.  The  heading  information 
leads  to  a  trajectory  (B)  closer  to  (C),  than  (A)  is. 


A2.1.5  Collaborative  work  on  further  algorithm  development 

Motivated  by  the  scope  of  our  SERDP  project,  exploratory  work  on  image  processing  algorithms 
was  carried  in  collaboration  with  the  University  of  Girona,  in  Spain.  Several  publications 
acknowledging  SERDP  resulted  from  this  effort.  Given  the  exploratory  nature  of  this  research, 
the  outcomes  were  not  intended  to  be  incorporated  into  the  software  deliverables.  Nonetheless  it 
resulted  in  several  algorithmic  improvements  that  benefitted  the  final  software  deliverables. 

The  exploratory  work  focused  on  three  topics:  1)  Selection  of  best  candidates  for  image 
matching,  2)  Automatic  change  detection  and  3)  Extension  of  the  graph-cut  blending  to  3D 
surfaces. 

A2. 1.5.1  Selection  of  best  candidates  for  image  matching 

Recent  advances  in  image  matching  methods,  such  as  the  SIFT,  allow  for  registering  pairs  of 
images  in  the  absence  of  prior  information  on  orientation,  scale  or  overlap  between  images. 
Evaluating  the  benefit  of  potential  matches  between  images  before  the  actual  matching  operation 
is  an  important  issue  that  has  not  yet  been  fully  addressed  in  the  literature  and  which  can  have  a 
large  impact  in  the  computational  efficiency  of  batch  mosaic  algorithms.  A  new  approach  for 
solving  this  issue  was  investigated  using  the  Observation  Mutual  Information  as  a  criterion  to 
evaluate  the  benefit.  This  allows  for  ranking  and  ordering  a  list  of  potential  matches  in  order  to 
make  the  loop-closing  process  more  efficient.  The  criterion  was  compared  against  other 
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proposed  strategies  and  results  are  presented  using  underwater  imagery.  This  work  was  presented 
at  a  conference  and  published  as  the  following  paper: 

Elibol,  A.,  N.  Gracias  and  R.  Garcia  (2009)  Match  Selection  in  Batch  Mosaicing  using  Mutual 
Information.  Proc.  of  the  4th  Iberian  Conference  on  Pattern  Recognition  and  Image  Analysis 
(IbPRIA2009),  Povoa  do  Varzim,  Portugal,  June  2009 

An  outcome  of  this  work  was  the  development  of  a  fast  method  to  find  the  most  similar  image 
pairs,  using  SIFT  descriptors  (Lowe  2004).  The  most  similar  image  pairs  are  matched  first.  Since 
the  image  matching  step  is  the  most  time  consuming  part  of  all  the  mosaicing  process,  the  use  of 
this  similarity  criterion  results  in  a  significant  saving  in  computation  time.  The  method  was 
implemented  in  multithreaded  C  and  incorporated  in  the  project  deliverables. 


A2.1.5.2  Automated  change  detection 

The  automated  detection  of  changes  between  images  of  the  same  area  of  the  benthos  is  of  great 
interest  for  improving  the  speed  of  environmental  monitoring.  However,  the  existing  algorithms 
for  change  detection  were  developed  mainly  for  aerial  surveying  and  require  highly  accurate 
registration  in  3D.  This  requirement  has  precluded  their  use  in  underwater  imagery. 

To  deal  with  this  problem,  a  new  approach  was  developed,  based  on  the  creation  of  3D  models 
of  small  areas.  Using  image-to-3D-model  registration  and  local  photometric  adjustment,  it 
attained  the  required  levels  for  accuracy  in  geometric  and  photometric  registration.  This  work  is 
the  first  successful  attempt  to  detect  changes  in  underwater  image  sets,  of  small  coral  reef 
patches.  It  was  published  and  presented  at  the  leading  conference  in  ocean  engineering  and 
published  in  the  following  paper: 

Delaunoy,  O,  N.  Gracias,  R.  Garcia,  (2008)  Towards  detecting  changes  in  underwater  image 
sequences,  Proceedings  of  the  IEEE  /  MTS  Oceans  2008  conference.  Kobe,  Japan,  April  2008. 


A2.1.5.3  Extension  of  the  graph-cut  blending  to  3D  surfaces 

The  method  for  efficient  image  blending  (Gracias  et  al,  2009),  described  above  as  part  of  the 
enhancement  of  the  2d  mosaicing  algorithm,  was  extended  for  blending  multiple  images  over  a 
3D  surface  for  creating  ortho-mosaics.  As  before,  the  objective  was  to  produce  seamless  image 
blending  that  preserves  the  clarity  of  object  textures,  even  in  the  presence  of  illumination 
changes  and  image  misalignments. 

The  blending  is  performed  by  optimizing  two  distinct  geometric  and  photometric  criteria.  A 
meaningful  geometric  criterion  was  considered  to  select  the  images  where  the  camera  position 
and  orientation  are  the  best  to  provide  texture  to  each  triangle  in  the  3D  model.  This  criterion 
provides  a  balance  between  choosing  nearby  cameras  and  reducing  perspective  distortions 
induced  by  the  slanting.  The  photometric  criterion  minimizes  the  visibility  of  the  seams  that 
occur  at  the  boundaries  of  neighboring  image  contributions.  The  balance  between  these 
independent  and  conflicting  criteria  is  obtained  by  graphcut  optimization. 
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Figure  A26  provides  an  example  of  the  blending  over  a  3D  surface  for  creating  an  ortho-mosaic. 
This  extended  blending  method  was  published  in  the  Journal  of  Field  Robotics  (Nicosevici  et  al 
2009). 


Figure  A26.  Comparison  of  blending  over  a  3D  surface,  obtained  using  a  simple  geometric  criterion  (a)  and 
the  improved  graph-cut  blending  method  extended  to  3D,  which  combines  geometric  and  photometric  criteria 
(b).  The  seams  visible  in  the  middle  and  bottom  of  (a)  are  eliminated  in  (b).  An  orthomosaic  using  the 
improved  method  is  presented  in  (c). 

A2.2  Mosaic  Products 

Between  2003  and  2009,  numerous  reef  sites  were  imaged  with  our  2D  mosaicing  technology. 
These  surveys  resulted  in  the  production  of  93  mosaic  images  for  use  in  traditional  and  non- 
traditional  monitoring  applications.  These  mosaics  are  presented  in  Appendix  2,  with 
information  on  ground  resolution  (pixel  size)  and  total  benthic  area. 

A2.3  Analysis  of  mosaics 
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A2.3.1  Geometric  analysis  of  landscape  mosaics 

An  important  aspect  in  evaluating  the  utility  of  the  2D  mosaics  is  to  determine  their  geometric 
accuracy.  To  some  extent,  our  current  mosaicing  algorithm  can  successfully  cope  with  violations 
of  the  assumption  that  the  bottom  is  static  and  flat.  However,  significant  topography  can 
contribute  to  geometric  distortions.  The  amount  of  distortion  needs  to  be  quantified  for 
ecological  analysis  involving  spatial  measurements.  In  order  to  assess  geometric  accuracy,  we 
have  established  a  method  and  performed  the  analysis  over  our  two  test  sites. 

The  method  involves  two  steps. 

1 .  In  the  first  step,  divers  establish  the  metric  positions  (XY  coordinates)  of  a  set  of  markers 
placed  on  the  sea  floor.  Once  established,  the  location  of  these  markers  is  considered 
ground-truth.  Since  it  is  difficult  to  directly  measure  the  XY  coordinates  of  points 
underwater,  the  creation  of  the  ground-truth  data  is  done  indirectly  using  a  network  of 
distance  measurements  between  points.  Field  data  for  measuring  these  distances  were 
collected  by  divers. 

2.  In  the  second  step,  the  locations  of  the  markers  as  measured  from  the  mosaic  are 
compared  with  diver  ground  truth.  To  do  this,  the  mosaic  image  has  to  be  georeferenced. 
The  georeferencing  is  done  by  computing  the  coordinate  mapping  between  the  mosaic 
image  and  the  world  metric  frame,  using  a  subset  (or  all)  of  the  ground-truth.  After  this, 
the  image  locations  of  the  markers  are  projected  onto  the  world  coordinate  frame.  An 
error  vector  is  defined  as  the  XY  differences  of  the  projections  with  the  ground-truth.  As 
measures  of  accuracy,  we  consider  (1)  the  standard  deviation  of  the  error  components, 
and  (2)  the  maximum  error  magnitude. 

An  initial  accuracy  assessment  was  performed  on  a  mosaic  from  the  Key  Biscayne  site  using 
video  data  acquired  in  June  2004.  A  ground-truth  set  of  24  positions  was  estimated.  The 
residuals  approximate  a  Normal  distribution  with  2  cm  standard  deviation,  which  is  consistent 
with  the  measurement  noise.  A  subset  of  20  marker  points,  visible  on  the  mosaic  (Figure  All), 
was  used  both  for  georeferencing  and  error  measurements.  The  error  vector  presents  a  standard 
deviation  of  5.1  cm.  The  maximum  error  magnitude  is  14.2  cm.  The  histogram  of  the  error  vector 
is  shown  in  Figure  A28. 
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Figure  A27.  Mosaic  image  from  the  Key  Biscayne  site  with  selected  marker  positions. 


Residuals  (m) 


Figure  A28.  Histogram  of  the  error  components,  with  superimposed  normal  distribution  fit. 
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Geometric  accuracy  analysis  of  the  Key  Largo  site  used  22  visible  ground  truth  points  and 
produced  comparable  results.  The  error  standard  deviation  is  7.7  cm  and  the  maximum  error 
magnitude  is  18  cm. 

Two  factors  influence  the  accuracy  and  applicability  of  this  analysis.  One  is  the  fact  that  the 
ground-truth  positions  are  obtained  in  2D,  thus  assuming  constant  elevation.  Although  care  was 
taken  in  choosing  locations  for  the  markers  that  were  approximately  on  the  main  plane  of  the 
terrain,  there  are  always  elevation  differences  that  are  not  accounted  for.  The  other  is  the  number 
of  markers  and  the  area  covered.  Current  results  correspond  to  around  20  markers  over  5x5  meter 
areas. 

A2.3.2  Ecological  analysis  of  landscape  mosaics 

A  subset  of  our  existing  mosaics  was  selected  to  evaluate  the  utility  of  the  mosaicing  technique 
in  coral  reef  monitoring  applications.  The  performance  of  the  mosaics  was  assessed  in  terms  of 
the  ability  to  1)  produce  spatially  accurate  coral  reef  scenes,  2)  extract  percent  cover 
measurements  of  major  benthic  categories,  3)  accurately  measure  colony  size  from  video 
mosaics,  and  4)  detect  changes  in  benthic  composition  over  time.  These  measures  of  reef 
condition  were  measured  by  divers  and  then  compared  directly  to  the  same  indicators  obtained 
from  mosaics  created  with  video  sequences  collected  at  the  same  time.  The  indicators  measured 
by  a  single  diver  (D.  Lirman)  were  used  as  the  standard  against  which  all  other  measurements 
were  compared. 


A2.3.2.1  Mosaic  Selection 

Three  mosaics  were  used  in  the  monitoring  assessment.  The  three  mosaics  are  of  the  same  reef 
plot;  they  were  collected  using  different  survey  platforms  and  represent  a  range  of  altitudes  and 
ground  resolution  (Table  A4). 

Table  A4.  Description  of  the  three  mosaics  used  in  the  monitoring  assessment.  Each  of  the  mosaics  was 
created  from  video  taken  at  the  Brooke’s  Reef  site  in  the  northern  Florida  Reef  Tract  (depth  =  7-10  m). 


Survey 

Date 

Survey  Platform 

(Camera  Resolution) 

Altitude 

Area 

Covered 

Ground 

Resolution 

1 

June  04 

Diver 

(720  x  530  pixels) 

2  m 

53  m2 

3.0  mm/pixel 

2 

April  05 

ROV 

(1024  x  768  pixels) 

2.5  m 

400  m2 

2. 5-3.0  mm/pixel 

3 

April  05 

ROV 

(1024  x  768  pixels) 

1.5  m 

45  m2 

1 .4  mm/pixel 

For  the  first  mosaic  (Figure  A29),  video  footage  was  acquired  in  June  2004  using  a  Sony  TVR 
900  DV  camcorder  in  an  underwater  camera  housing.  This  first  survey  is  included  to  illustrate 
that  the  mosaicing  algorithm  can  produce  geometrically  accurate  mosaics  from  a  standard,  low- 
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cost,  handheld  camera.  The  mosaic  was  created  from  365  key-frames  selected  using  a  criterion  of 
75%  overlap  between  consecutive  images.  Frame  resolution  is  720  x  530  pixels. 


Figure  A29.  Video  mosaic  created  from  hand  held  video  images  taken  at  the  Brooke’s  Reef  site  in  June  2004. 


For  all  mosaics,  the  camera  followed  a  “lawnmower  pattern”  of  side-by-side  strips, 
complemented  by  the  same  pattern  rotated  90°  to  ensure  full  coverage  of  the  area  and  high 
superposition  among  the  strips.  The  colors  on  all  mosaics  were  adjusted  by  manually  selecting 
both  a  white  and  a  black  reference  and  linearly  interpolating  the  red,  green,  and  blue  intensities. 
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The  algorithms  were  coded  in  Matlab  6.2,  and  the  overall  processing  took  between  6-12  hours 
per  mosaic  using  a  3.0GHz  PC. 

For  the  second  and  third  mosaics  (Figures  A30  and  A31),  video  was  collected  using  a  Pt  Grey 
Flea  camera  mounted  on  a  Phantom  remotely  operated  vehicle  (ROV)  (Xu,  2000)  in  April  2005. 
Both  high  and  low  altitude  datasets  were  used  for  assessing  ecological  indices.  The  cameras 
were  internally  calibrated  to  reduce  image  distortion  from  the  lens  and  housing  (Bouguet,  2002). 
Frame  resolution  is  1024  x  768  pixels. 


Figure  A30.  ROV  Video  Mosaic  created  from  a  high  altitude  (2.5m)  pass  of  the  Brooke’s  Reef  site  in  April 
2005. 

For  the  second  mosaic  (Figure  A30),  496  key  frames  were  selected  out  of  the  complete  set  of 
5061  images,  using  72%  overlap.  The  registration  parameters  for  the  non  key- frames  were 
obtained  by  linear  adjustment  of  the  sequential  matching,  constrained  by  the  registration 
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parameters  of  the  two  closest  key- frames.  For  the  third  mosaic  (Figure  A31),  872  key  frames 
were  selected  from  a  set  of  3439  images  with  a  75%  overlap  criterion. 


Figure  A31.  ROV  based  video  mosaic  created  from  a  low  altitude  pass  (1.5m)  of  the  Brooke’s  Reef  site  in 
April  2005. 
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A2.3.2.2  Spatial  Accuracy 

The  geometric  accuracy  of  three  of  the  2D  mosaics  was  analyzed  by  comparing  the  locations  of 
points  measured  from  the  mosaics  with  diver  measurements  of  the  locations  of  the  same  points. 
Distortion  indicators  showed  an  improvement  in  spatial  accuracy  (i.e.,  decreases  in  the  standard 
deviations  of  the  residuals  and  maximum  distance  errors)  going  from  video  collected  by  a  diver 
holding  a  digital  camcorder  (first  mosaic)  to  video  collected  by  a  high-resolution  camera 
mounted  on  the  ROV  (second  mosaic).  Distortion  indicators  did  not  improve  with  increased 
image  resolution  (i.e.  a  smaller  pixel  size),  however,  because  to  obtain  smaller  pixel  size  requires 
image  acquisition  closer  to  the  bottom,  thereby  increasing  the  distortions  due  to  3D  structure  of 
the  scene.  Standard  deviations  of  the  residuals  were  5.1,  3.9,  and  5.5  cm,  while  maximum 
distance  errors  were  12.9,  10.7,  and  13.5  for  the  first,  second,  and  third  mosaics  respectively. 

The  ability  to  extract  accurate  distance  measurements  from  the  mosaics  was  evidenced  by  the 
low  values  calculated  for  the  distortion  indicators.  Moreover,  the  spatial  accuracy  of  the  video 
mosaics  presented  here  was  similar  or  lower  than  the  measurement  uncertainty  of  diver 
measurements,  which  typically  exhibits  a  standard  deviation  of  5  cm  (Holt,  2000).  While  an 
improvement  in  camera  resolution  resulted  in  a  reduction  in  spatial  distortion,  the  higher 
distortion  of  the  low-altitude  mosaic  highlighted  a  present  limitation  of  the  mosaic  algorithm. 
The  sources  that  contribute  to  spatial  distortions  in  mosaics  include:  (1)  departures  from  the 
model  assumption  of  a  flat  environment;  (2)  amount  of  superposition  among  strips  during  the 
acquisition;  (3)  limited  visibility  underwater;  (4)  limited  resolution  of  the  imaging  sensors;  and 
(5)  limited  accuracy  of  the  image  matching  algorithm.  The  higher  distortion  recorded  for  the 
third  mosaic,  collected  closest  to  the  bottom,  can  be  likely  attributed  to  the  fact  that  the  planarity 
assumptions  were  clearly  violated  at  the  low  altitude  at  which  the  video  sequence  was  collected. 


A2.3.2.3  Percent  Cover 

The  percent  cover  of  the  dominant  benthic  organisms  measured  by  trained  divers  in  situ 
compared  favorably  with  those  measured  directly  from  the  video  mosaics.  Five  out  of  the  eight 
categories  chosen  (hard  corals,  octocorals,  Palythoa,  sponges,  and  sand)  showed  no  significant 
differences  between  diver  based  surveys  and  high  and  low  altitude  video  mosaics  in  terms  of 
percent  cover  (Table  A5,  p  >  0.05).  The  remaining  three  categories,  all  corresponding  to 
functional  forms  of  reef  macroalgae  (erect  macroalgae,  turf,  and  crustose  coralline  algae)  did 
show  significant  differences  among  survey  methodologies  (p  <  0.05).  However,  when  these 
categories  are  grouped  together  into  a  single  macroalgae  group,  no  significant  differences  were 
found  among  survey  methodologies  (p  >  0.05). 


48 


RC-1333  Final  Report  to  SERDP  2010 


R.P.Reid 


Table  A5.  Mean  cover  (±  S.E.M.)  of  the  different  benthic  categories  surveyed  by  divers  and  measured  from 
video  mosaics  from  a  reef  site  in  the  northern  Florida  Reef  Tract  (depth  =  7-10  m).  Divers  surveyed  twenty- 
five  0.25  m2  quadrats.  For  comparison,  a  subset  of  twenty-five  quadrats  (0.25  m2)  was  sampled  at  random 
from  the  video  mosaics  collected  at  2  different  resolutions.  High-resolution  mosaics  were  collected  at  a 
distance  of  1.5  m  to  the  bottom  (2.5-3.0  mm/pixel).  Low-resolution  mosaics  were  collected  at  a  distance  of  2.5 
m  to  the  bottom  (1.4  mm/pixel).  CCA  =  Crustose  Coralline  Algae,  p-values  from  a  Kruskal-Wallis  test. 


Benthic 

Categories 

Diver 

Mosaic  -  High 
Resolution 

Mosaic  -  Low 
Resolution 

P 

Stony 

Corals 

1.4  (0.5) 

2.0  (0.7) 

1.8  (1.0) 

0.6 

Octocorals 

7.5  (2.6) 

6.2  (1.6) 

4.7  (1.6) 

0.6 

Macroalgae 

38.1  (3.4) 

31.7  (3.0) 

21.2  (3.1) 

<0.01 

CCA 

1.1  (0.4) 

0.3  (0.2) 

0 

0.02 

Sponges 

3.4  (1.2) 

12.9(1.9) 

13.6(1.9) 

<0.01 

Palythoa 

4.2  (2.6) 

1.2  (0.5) 

2.7  (1.7) 

0.3 

Sand 

5.8  (2.0) 

9.2  (2.0) 

7.5  (1.7) 

0.6 

Turf 

38.9  (2.9) 

36.5  (3.0) 

41.6  (3.9) 

0.3 

As  an  additional  measurement  of  coral  cover,  the  boundaries  of  all  stony  corals  found  within  the 
area  imaged  by  the  low-altitude  mosaic  were  digitized  and  analyzed  using  the  “particle  analysis” 
feature  in  the  ImageJ  software  that  calculates  the  total  area  of  polygon  features  within  an  area  of 
interest.  The  coral  cover  value  obtained  by  digitizing  the  boundaries  of  all  of  the  coral  colonies 
within  the  area  imaged  by  the  high-resolution  mosaic  (2.8  %)  was  within  the  95%  confidence 
intervals  of  the  values  obtained  by  divers  (1.4%)  and  from  video  mosaics  (2.0  and  1.8%)  using 
the  point-count  method  (Table  A5). 

Finally,  the  abundance  of  juvenile  corals  (<  4  cm  in  diameter)  measured  by  divers  within  benthic 
quadrats  was  compared  to  the  abundance  of  juvenile  corals  measured  from  the  mosaic  tiles. 
While  the  mean  abundance  of  juvenile  corals  (<  4  cm  in  diameter)  documented  by  divers  during 
visual  surveys  were  1.1  and  1.4  juveniles  m'2,  no  juvenile  corals  were  detected  from  the  mosaics. 
We  conclude  that  a  major  limitation  of  video-mosaic  surveys  is  the  ability  to  detect  and  identify 
juvenile  corals  (<  4  cm  in  diameter).  This  limitation  of  video  mosaic  surveys  is  usually  due  to 
the  fact  that  these  small  corals  are  often  found  on  cryptic  habitats  and  can  only  be  seen  in  visual 
surveys  where  the  observer  can  shift  the  angle  of  view.  Future  improvements  in  camera 
resolution  should  enhance  the  ability  to  detect  juvenile  corals  from  2D  mosaics  and  facilitate  the 
classification  of  additional  benthic  categories.  These  issues  have  been  mostly  addressed  with  the 
implementation  of  the  second-generation  mosaicing  system  and  its  benefits  are  discussed  in 
section  A2.3.3. 
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Figure  A32.  Abundance,  spatial  distribution,  and  sizes  of  stony  corals  obtained  from  a  low-altitude  (1.5  m 
from  the  bottom)  high-resolution  (1.4  mm/pixel)  video  mosaic  (a).  The  boundaries  of  each  coral  colony  (b) 
were  digitized  and  the  benthic  coverage  of  stony  corals  was  measured  using  the  ImageJ  software.  The  coral 
cover  obtained  by  this  method  (2.8  %)  was  within  the  95%  confidence  intervals  of  the  values  obtained  by 
divers  (1.4%)  and  from  video  mosaics  (2.0  and  1.8%)  using  the  point-count  method. 


A2.3.2.4  Colony  Size 

To  determine  the  accuracy  of  diver  surveys  and  video  mosaics  to  estimate  coral  colony  size,  the 
differences  between  the  values  obtained  by  Lirman  and  those  obtained  by  a  second  diver  (B. 
Gintert),  or  directly  from  the  video  mosaics  were  measured.  Accuracy  of  the  size  measurements 
was  ascertained  by  calculating  two  measurements  of  error  as  described  by  Harvey  et  al.  (2000): 

Absolute  Error  =  AE  =  (  I  Diver  1  -  Diver  2| )  and  (  I  Diver  1  -  Mosaicl ) 

Relative  Absolute  Error  =  RAE  =  [( I  Diver  1  -  Diver  2| )  /  Diver  1] 

and  [( |  Diver  1  -  Mosaicl )  /  Diver  1] 

To  compare  the  size  data  collected  by  divers  and  mosaics,  an  ANOVA  with  two  factors,  survey 
method  and  coral  size  category,  was  performed  using  the  AE  values.  When  the  accuracy  of  the 
two  methods  was  compared  using  the  AE,  significant  differences  were  found  among  the  size 
categories,  with  AE  increasing  with  colony  size  and  height  (ANOVA,  p  <  0.01)  (Table  A5). 
However,  no  significant  differences  were  documented  based  on  survey  method  (ANOVA,  p  > 
0.05).  The  mosaic-diver  deviations  were  generally  higher  for  the  diver-mosaic  comparisons  for 
small  colonies  (<  20  cm)  but  higher  for  diver-diver  comparisons  for  larger  colonies  (Table  A5). 

Table  A6.  Comparison  of  coral  size  measurements  between:  (1)  two  divers  measuring  the  same  colonies;  and 
(2)  between  diver  measurements  and  measurements  of  the  same  colonies  obtained  directly  from  the  video 
mosaics.  AEi  =  Absolute  Error  =  (|  Diver  1  -  Diver  2| ),  RAEi  =  Relative  Absolute  Error  =  [( |  Diver  1  -  Diver 
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2| )  /  Diver  1  ].  AE2  =  Absolute  Error  =  (|  Diver  1  -  Mosaicl ),  RAE2  =  Relative  Absolute  Error  =  [( |  Diver  1  - 
Mosaicl )  /  Diver  1  ].  Measurements  taken  by  Diver  1  (Lirman)  were  considered  here  as  the  standard  against 
which  all  other  measurements  were  compared.  Values  reported  are  means  (+  S.D.). 


Coral  Sizes 
(cm) 

Diver-Diver  Comparison1 

Diver-Mosaic  Comparison 

AE! 

RAE! 

N 

CM 

LU 

< 

rae2 

N 

<10 

0.7  (0.3) 

8.9 

9 

1.6  (0.4) 

21.0 

22 

10-20 

1.9  (0.7) 

10.6 

15 

2.5  (0.4) 

16.5 

45 

>20-30 

4.8  (1.2) 

17.7 

7 

3.4  (0.8) 

14.2 

19 

>30-80 

5.4  (2.7) 

11.1 

7 

5.6  (1.4) 

13.1 

20 

The  capability  of  identifying  individual  coral  colonies  and  measuring  their  size  directly  from 
each  mosaic  is  one  of  the  most  important  benefits  of  2D  video  mosaics.  While  the  accuracy  of 
the  mosaic  measurements  relative  to  the  diver-based  measurements  was  influenced  by  colony 
size,  these  patterns  result  from  the  difficulty  that  divers  commonly  encounter  while  trying  to 
measure  coral  colonies  in  the  field.  Colony  boundaries  are  easily  distinguished  in  small  (<  20 
cm)  colonies  that  commonly  exhibit  circular  shapes,  but  larger  colonies  with  irregular  shapes 
pose  a  challenge  for  divers  trying  to  delimit  live  tissue  boundaries.  Future  improvements  in  the 
3D  representation  of  benthic  mosaics,  such  as  the  work  presented  in  Section  B  (Negahdaripour 
and  Madjidi,  2003;  Negahdaripour  et  al.,  2005),  are  expected  to  substantially  improve  the 
accuracy  of  colony  size  measured  from  mosaics,  particularly  for  larger  colonies  with  more 
complex  topographies.  The  analysis  of  mosaics  constructed  over  two  spatial  dimensions  has 
highlighted  several  advantages  over  strip  mosaics  constructed  along  a  single  spatial  dimension. 
For  example,  the  sizes  of  coral  colonies  were  accurately  measured  from  two-dimensional 
mosaics,  even  though  they  are  typically  hard  to  acquire  from  one-dimensional  mosaics  where 
only  the  smallest  coral  colonies  are  completely  imaged  along  a  single  transect. 


A2.3.2.5  Change  Detection 

An  additional  benefit  of  using  spatially  accurate  mosaics  is  our  ability  to  easily  return  to  a  fixed 
point  within  the  resulting  image,  making  it  easy  to  follow  single  colonies  over  time.  This  ability 
to  track  changes  in  single  colonies  may  help  to  identify  the  effects  of  disease  or  bleaching  more 
quickly  and  precisely  than  if  using  percent  cover  alone.  The  removal  of  coral  colonies  or  other 
benthic  organisms  and  changes  in  the  composition  of  the  substrate  can  be  easily  discerned  by 
looking  at  the  same  section  of  the  reef  in  a  time  series  of  video  mosaics  (Figure  A33). 
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Figure  A33.  Example  of  co-registered  mosaic  sub-sections  or  tiles  (1  m  x  1  m)  used  to  assess  patterns  of 
change  in  the  abundance  and  distribution  of  benthic  organisms  between  2004  (a)  and  2005  (b).  The  box 
highlights  the  removal  or  mortality  of  a  small  (<  10  cm  in  diameter)  coral  colony  between  surveys. 

Using  this  method,  the  mortality  or  removal  of  four  coral  colonies  (out  of  50  colonies)  was 
documented  between  2004-2005  (Mosaics  1  and  3)  from  an  area  of  approximately  16  m2  (Figure 
A33).  Moreover,  two-dimensional  imagery  from  repeated  surveys  was  accurately  referenced  to 
assist  with  change-detection,  unlike  linear  transects  that  are  exceedingly  difficult  to  duplicate 
precisely  over  time. 

Previous  research  on  the  design  of  field  programs  aimed  at  documenting  patterns  of  change  in 
benthic  resources  over  time  has  highlighted  the  increased  statistical  power  gained  by  surveying 
precise  specific  locations  repeatedly  compared  to  the  survey  of  random  locations  (Van  de  Meer, 
1997;  Ryan  and  Heyward,  2003).  The  demarcation  of  permanent  plots  on  hard  benthic  substrate 
is  commonly  achieved  by  attaching  pipes  or  nails  on  the  bottom,  and  the  number  of  markers 
needed  to  mark  multiple  colonies,  quadrats,  or  transects  at  a  given  site  can  be  quite  large.  Video 
mosaics  provide  an  alternative  to  these  labor-intensive  methods.  By  placing  a  limited  number  of 
permanent  markers  to  provide  a  reference  frame  within  each  video  mosaic  (only  four  permanent 
markers  were  used  in  this  study  to  accurately  survey  an  area  of  400  m2),  the  technique  described 
in  this  study  can  reduce  significantly  the  bottom-time  needed  to  collect  ecological  information  in 
the  field.  Moreover,  by  providing  the  ability  to  survey  specific  sub-plots  repeatedly  within  a 
larger  area  of  the  benthos,  video  mosaics  provide  increased  statistical  power  to  detect  small 
changes  in  abundance,  cover,  and  size  of  benthic  organisms.  However,  a  trade-off  exists  between 
within-site  precision  and  the  ability  to  survey  large  areas,  making  the  video  mosaic  technique  an 
ideal  method  to  survey  areas  <  500  m2  but  impractical  for  documenting  changes  in  the  extent  and 
condition  of  benthic  resources  at  larger  spatial  scales  within  a  single  image.  Of  course  multiple 
mosaics  can  be  used  to  statistically  sample  larger  areas.  It  is  expected  that  further  improvements 
in  the  mosaicing  algorithms  combined  with  the  use  of  improved  positioning  systems  (e.g., 
acoustic  transponder  networks)  will  make  construction  of  even  larger  2-D  mosaics  practical  at  in 
the  near  future. 
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A2.3.3  Benefits  of  second-generation  mosaic  technology  to  ecological  analysis 

First-generation  video  mosaics  were  an  innovative  reef  survey  technology  that  provided  large- 
scale  (up  to  400  m2),  spatially  accurate,  high-resolution  images  of  the  reef  benthos  without 
extensive  survey  times  or  a  need  for  scientific  divers.  Despite  these  advances,  first-generation 
mosaic  products  were  insufficient  for  species-level  identification  of  many  benthic  taxa,  thereby 
limiting  the  monitoring  potential  of  the  technique.  The  second-generation  mosaic  survey 
technology  that  was  developed  in  2007  integrates  high-resolution  still-image  acquisition  with 
high-definition  video  surveys  of  the  reef  benthos  (see  section  A2.1.4  for  methods). 

The  first-generation  landscape  mosaics  were  created  from  video  taken  with  either  a  Sony  TRV 
900  DV  camera  hand-held  by  a  diver,  or  a  Point  Grey  Research  Flea  camera  mounted  on  a 
Remotely  Operated  Vehicle.  These  two  camera  systems  provided  spatial  resolution  ranging  from 
one  to  four  mm/pixel,  which  was  sufficient  to  identify  taxonomic  groups,  such  as  stony  corals, 
sponges,  gorgonians,  macroalgae  and  sand,  but  was  not  fine  enough  to  allow  species-level 
identification  in  most  cases  (Lirman  et  al.  2007).  Organisms  smaller  than  5  cm  in  diameter  were 
typically  not  detectable  using  first-generation  mosaic  products  (Lirman  et  al.  2007). 

To  overcome  the  resolution  limits  of  the  first- generation  mosaic  survey  technology  while 
maintaining  the  high  frame  rate  and  altitude  requirements  of  the  mosaic  processing  algorithms,  a 
two-camera  system  was  adopted.  This  second-generation  mosaic  imaging  system  combines 
information  from  (1)  a  high-definition  video  (HDV)  camera  capable  of  full  video  frame  rate  (30 
fps)  and  (2)  a  digital  still  camera  capable  of  acquiring  very  high-resolution  images  (10  MP)  at 
low  capture  rates  (1  fps). 

In  the  field,  the  second-generation  imaging  system  is  deployed  with  both  cameras  in  a  down¬ 
looking  position.  Cameras  are  swum  by  a  single  diver  2  m  above  the  reef  area-of-interest  in  a 
double  lawn-mower  pattern  as  described  previously  (Lirman  et  al.  2007). 

Data  processing  of  the  second-generation  mosaic  data  is  described  previously  (section  A2. 1.4.1). 
Display  software  was  developed  that  allows  users  to  select  any  point  within  the  landscape  mosaic 
produced  by  the  second-generation  imaging  technology  and  retrieve  the  corresponding  10  MP 
still  image  and  corresponding  video  frame  of  the  area  surrounding  that  point  (Reid  et  al,  2007). 

Testing  of  the  second-generation  mosaicing  system  continued  in  2008,  with  the  mapping  of  a 
portion  of  Brooke’s  Reef,  FL  (25°  40.508’N,  80°  5.908’W)  in  June.  The  landscape  mosaic 
generated  from  the  high-definition  video  of  the  second-generation  system  covered  156  m2 
(Figure  A34A).  The  first-generation  landscape  mosaic  of  Brooke's  Reef  taken  in  June  2004  has  a 
spatial  resolution  of  3  mm/pixel  (Figure  A34B),  whereas  the  second  generation  version  has  a 
spatial  resolution  of  2  mm/pixel  (Figure  A34C).  The  second-generation  still  camera  images  were 
acquired  at  a  frame  rate  of  1  fps,  resulting  in  approximately  1,800  10  MP  images  of  the  reef 
benthos.  The  1,800  still  images  were  then  matched  to  key  video  frames.  The  average  spatial 
resolution  of  the  still  images  from  the  second-generation  system  was  0.4  mm/pixel,  almost  an 
order  of  magnitude  improvement  over  the  video  of  the  first  generation  system  (Figure  A34D,  E). 
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Figure  A34.  (A)  A  second-generation  video  mosaic  from  Brooke’s  Reef,  FL  taken  in  June  2008.  Images  of  a 
Meandrina  meandrites  colony  (shown  in  A)  selected  from  mosaic  surveys  created  using:  (B)  a  Sony  DV 
camcorder  in  June  2004,  (C)  high  definition  Sony  camcorder  from  June  2008,  and  (D-E)  10  MP  Nikon  D200 
still  camera.  All  images  were  acquired  at  a  distance  of  2  m  from  the  benthos.  The  combination  of  high 
resolution  still  images  with  video  acquisition  provides  increased  reef  health  information  such  as  (D)  tissue 
partial  mortality  and  (E)  cyanobacterial,  macroalgal,  and  sponge  interactions. 
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The  2  mm/pixel  spatial  resolution  of  the  second-  generation  landscape  mosaic,  produced  from 
the  high-definition  video,  enabled  improved  analysis  of  the  mosaic.  Boundaries  between 
sponges,  corals  and  gorgonians  were  clearer  and  large  colony  identification  (>  25  cm)  was 
possible  for  most  Caribbean  species  (Figure  A34C). 

Analysis  of  the  10  MP  still  images  from  the  second-generation  system  provided  high  taxonomic 
resolution.  The  sub-millimeter  resolution  of  the  still  images  allowed  for  species-level 
identification  of  colonies  as  small  as  3  cm  in  diameter.  In  addition,  macroalgal  groups  were 
identifiable  to  genus  with  species  level  identification  possible  for  macroalgae  with  obvious 
defining  characteristics  such  as  Halimeda  tuna  and  H.  opuntia.  Coral  colony  health  information 
such  as  partial  mortality  boundaries  and  evidence  of  bleaching  and  disease  were  all  recognizable 
using  the  still  images.  Small-scale  indicators  of  reef  health  such  as  cyanobacteria,  macroalgal, 
and  sponge  competition  were  also  visible  using  high-resolution  still  data  (Figure  A34D,  E). 

The  first-generation  video  mosaics  demonstrated  that  mosaic-based  monitoring  could  be 
accomplished  without  trained,  expert  scientific  divers,  long  dive  times,  or  extensive  tagging  of 
coral  colonies.  This  technique  also  allowed  users  to  assess  basic  reef  health  indices  and  monitor 
coral  colonies  using  a  large-scale  image-based  approach  that  limited  the  need  for  extensive  dive¬ 
time  and  provided  long  lasting  visual  information  on  the  state-of  the  reef  at  the  time  of  the 
survey  (Lirman  et  al.  2007). 

Second-generation  mosaics  retain  these  monitoring  strengths  while  providing  increased  benthic 
resolution  over  the  entire  survey  area;  enabling  species  level  information  for  most  coral  colonies. 
This  allows  greater  taxonomic  information  with  respect  to  percent  cover  and  diversity  indices 
than  was  available  from  first-generation  mosaics.  In  addition,  change  detection  analysis  using 
second-generation  mosaic  products  provides  information  on  conspicuous  indices  of  reef  health 
(such  as  bleaching  and  tissue  loss)  while  also  allowing  users  to  monitor  fine  scale  stressors  such 
as  macroalgal  and  cyanobacterial  interactions  and  predation  events  (Figure  A35).  This  represents 
a  significant  monitoring  improvement  over  first-generation  products  that  could  only  discern 
dramatic  indicators  of  colony  health  such  as  bleaching,  and  significant  tissue  loss  (Gleason  et  al. 
2007). 
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Figure  A35.  Example  of  change  detection  potential  using  second-generation  imagery.  High  resolution  still 
images  were  acquired  from  approximately  2  m  water  depth  during  video  mosaic  surveys.  Images  were 
automatically  matched  to  their  geographic  locations  within  the  survey  area  for  change  detection  analyses. 
Between  February  2008  (A)  and  September  2008  (B)  the  Porites  porites  (center)  and  Stephanocoenia 
intersepta  (right)  colonies  shown  above  have  undergone  considerable  paling.  Evidence  of  partial  mortality 
and  an  increase  of  cyanobacteria  within  the  P.  porites  colony  are  also  identifiable  by  comparing  the  survey 
images.  Small  indicators  of  coral  health  such  as  fish  bite  predation  are  also  clearly  present  on  both  the  P. 
porites  and  the  S.  intersepta  in  February  2008  and  can  be  followed  through  time  to  monitor  long-term 
impacts  (A). 

Although  many  monitoring  programs  take  pictures  of  the  reef  for  analysis  and  documentation 
purposes,  the  extensive  cataloging  and  database  management  required  in  the  laboratory  often 
limits  the  amount  of  photographic  data  acquired  during  a  survey.  The  automated  matching  of 
high  resolution  still  images  to  their  geographic  location  within  a  second-generation  mosaic 
removes  the  need  for  user-intensive  laboratory  cataloging  and  allows  the  archiving  of  several 
thousand  high  resolution  images  of  a  reef  area  with  minimal  user  input. 
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Image-based  techniques  for  reef  monitoring  have  an  excellent  archive  potential  that  is  vastly 
superior  to  written  notes  by  divers.  The  second-generation  mosaic  survey  technology  described 
here  provides  both  landscape  level  and  colony  level  (as  small  as  3  cm)  permanent  records  of  the 
state-of-the-reef  at  the  time  of  the  survey  without  increasing  the  dive-time  of  first-generation 
mosaic  products.  The  simultaneous  capture  of  both  of  these  scales  of  information  allows  users  to 
monitor  large-scale  changes  in  reef  communities  such  as  hurricane  or  ship  grounding  damage,  as 
well  as  small  scale  interactions  such  as  sponge  and  macroalgal  competition  using  the  same 
survey  design  and  the  same  raw  data  set. 

The  incorporation  of  high-resolution  still  imagery  into  the  second-generation  imaging  equipment 
and  mosaic  processing  resolved  the  species  identification  limitations  of  the  first-generation 
system.  The  ability  to  capture  high  resolution  images  over  100’s  of  square  meters  and 
georeference  these  images  within  video  mosaics  without  user  input  is  a  major  advantage  of  the 
landscape  mosaic  technology.  The  combination  of  large-scale  image  maps  and  high-resolution 
benthic  information  provides  a  unique  tool  that  maintains  the  strengths  of  most  traditional 
monitoring  methods,  such  as  the  ability  to  extract  indices  of  reef  health  (percent  cover,  coral 
colony  sizes,  and  diversity  indices),  while  also  providing  a  rapid  method  to  document  and  assess 
changes  at  both  large  and  small  scales  in  coral  communities  without  extensive  tagging.  This 
combined  with  the  extensive  archive  potential  inherent  to  this  technology  provides  a  significant 
advancement  in  coral  reef  community  monitoring  technology 

The  above  information  is  presented  in  the  following  paper: 

Gintert,  B.,  Gracias,  N.,  Gleason,  A.C.R.,  Lirman,  D.,  Dick,  M.,  Kramer,  P.,  Reid,  R.P.  (2008) 
Second-Generation  Landscape  Mosaics  of  Coral  Reefs.  Proceedings  of  the  11th  International 
Coral  Reef  Symposium.  Fort  Lauderdale,  FL,  July  7-11,  2008.  (In-press) 

A2.3.4  Recommendations  for  Field  Implementation 

The  collection,  processing,  and  analysis  of  landscape  video  mosaics  may  be  affected  by  water 
quality  parameters  at  the  time  of  collection.  In  addition,  because  of  the  mosaic  algorithm  that 
calculates  camera  trajectory  based  on  pixel  location  within  sequential  images,  camera  motion 
and  movement  of  objects  on  the  bottom  are  issues  of  concern. 

Over  the  past  six  years,  video  footage  has  been  collected  for  mosaic  development  under  a  wide 
variety  of  environmental  conditions.  The  range  of  conditions  and  the  quality  of  the  resulting 
mosaics  have  allowed  us  to  develop  a  set  of  guidelines  or  best  practices  for  the  collection  of  field 
data  that  can  be  use  to  determine  under  which  conditions  the  likelihood 

of  creating  a  good  mosaic  can  be  maximized  and,  more  importantly,  which  conditions  to  avoid  if 
possible. 

These  guidelines  are  as  follows: 

1 .  Environmental  conditions:  At  the  inception  of  project  SI  1333  two  fixed  monitoring  sites  were 
established  in  Florida  at  Brooke’s  Reef  and  Grecian  Patch.  These  two  sites  were 
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characterized  by  both  their  environmental  and  biological  characteristics  in  an  attempt  to 
parameterize  successful  data  collection  of  the  prototype  system  (Table  A7). 

Table  A7.  Biological  and  Environmental  characteristics  of  Brooke’s  Reef  and  Grecian  Patch  permanent 
monitoring  sites. 


BROOKE'S  REEF 

GRECIAN  PATCH 

REGION 

KEY  BISCAYNE 

KEY  LARGO 

HABITAT  TYPE 

PATCH  REEF 

PATCH  REEF 

LOCATION 

25  40,508  N,  80  5,908  W 

25  6.362  N,  80  19,086  W 

DEPTH  (m) 

7-10 

4-6 

EXPOSURE 

HIGH 

LOW 

SEDIMENTATION  (g/day) 

0.81  (1.13) 

0.03(0,01) 

HARD  CORALS  (%) 

5.4  (1.60) 

9.4  (2,1) 

SOFT  CORALS  (%) 

4.1  (0,9) 

11.4(2.6) 

SPONGES (%) 

10.1  (1.1) 

7,1  (IS) 

PALYTHOA  (%) 

4.9(19) 

4,4(17) 

MACROALGAE  (%} 

32.8  (2,3) 

25.3  (2,5) 

TURF  ALGAE  (%) 

18.2  (2,3) 

24.1  (3,3) 

CRUSTOSE  ALGAE  (%) 

8.8  (1.1) 

6.2  (1.7) 

SAND  (%) 

15.7(2.2) 

12,2(2,4) 

JUVENILE  CORALS  (n/m2) 

1.3  (0.6) 

1.6(09)  I 

N  OF  DIADEMA 

0 

0 

TOPOGRAPHY  (cm) 

108  (41) 

57(5) 

Visibility  and  water  clarity  were  different  at  the  two  sites.  The  large  tidal  influence  and 
proximity  to  Biscayne  Bay  at  Brooke’s  Reef  caused  visibility  to  be  generally  lower  and  mean 
sedimentation  to  be  more  than  20  times  higher  than  at  the  Grecian  Patch  site  (Table  A7). 
Visibility  at  the  Grecian  Patch  site  is  usually  excellent  and  only  after  severe  storms  does 
sediment  resuspension  result  in  reduced  water  clarity.  Despite  the  disparity  in  environmental 
characteristics  of  these  two  sites,  the  in-water  conditions  were  never  severe  enough  to 
prevent  mosaic  creation  at  these  fixed  sites.  Table  A8  gives  a  summary  of  environmental 
conditions  on  days  in  which  permanent  monitoring  sites  were  sampled. 

Table  A8.  Environmental  observations  on  dates  in  which  permanent  monitoring  sites  were  sampled. 
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Date 

Air  Temp 

(°C) 

Avg  wind 

speed  (knots) 

Barometric 
pressure  (mb) 

Practical 

Salinity 

3/30/2004 

21.3 

11.4 

1018.3 

24.2 

4/1/2004 

22.2 

17.3 

1014.4 

24.5 

4/6/2004 

21.5 

8.6 

1017.4 

24.1 

10/20/2004 

27.7 

7.8 

1015.1 

28.1 

3/5/2005 

19.7 

10.2 

1019.5 

23.0 

4/1/2005 

24.3 

14.2 

1016.1 

24.2 

5/9/2005 

23.4 

10.7 

1018.3 

25.2 

12/9/2005 

25.3 

14.3 

1020.8 

24.4 

5/9/2006 

26.1 

12.3 

1014.1 

26.4 

3/2/2007 

25.0 

12.4 

1012.2 

24.3 

6/16/2008 

28.5 

8.6 

1014.7 

28.2 

8/7/2008 

30.0 

5.0 

1016.1 

30.2 

8/11/2008 

28.5 

9.7 

1014.9 

29.3 

9/15/2009 

28.5 

9.3 

1013.7 

29.7 

In  general,  successful  mosaic  sampling  occurred  when  average  wind  speeds  were  below 
15knots.  The  one  exception  to  this  trend  occurred  on  4/1/2004  in  which  the  Brooke’s  Reef 
site  was  successfully  sampled  on  a  day  with  an  average  wind  speed  of  17.3  knots.  The  success 
of  this  mosaicing  is  attributed  to  the  greater  water  depth  and  decreased  dependence  on  surface 
wave  conditions  for  water  clarity  at  this  site. 

In  addition  to  the  permanent  monitoring  sites  established  in  Florida,  nearly  40  other  sites  were 
sampled  during  the  term  of  the  SI  1333  project.  Although  environmental  data  were  not 
collected  for  each  of  these  additional  sites  due  to  heavy  processing  costs  and  limitations  of 
collaborative  projects,  some  important  environmental  observations  were  extracted  from  non¬ 
permanent  sites  to  establish  optimal  working  conditions  for  the  prototype  system.  Of  the  data 
collected,  conditions  were  suitable  for  creating  mosaics  in  all  but  one  set  of  field  collections. 
During  a  collaborative  effort  with  the  University  of  Puerto  Rico,  in-water  properties  were 
encountered  that  prevented  mosaic  creation.  In  association  with  NOAA,  environmental 
parameters  for  the  study  area  near  Media  Luna  Reef,  Puerto  Rico  were  obtained  and  analyzed 
to  provide  optimum  environmental  parameters  for  mosaic  acquisition. 

On  December  11,  2007  an  out  of  season  tropical  storm  (Tropical  Storm  Olga)  passed  over  the 
island  of  Puerto  Rico.  Due  to  inflexible  travel  constraints  mosaic  data  was  collected  on 
December  12th  13th  at  4  reef  sites  near  the  Media  Luna  Reef  Integrated  Coral  Observation 
Network  (ICON)  station  in  Puerto  Rico. 

Due  to  the  passage  of  the  storm  the  previous  day,  water  clarity  (as  measured  by  the  diffuse 
light  attenuation  coefficient  Kd)  was  very  low.  Kd  varied  between  0.3  and  0.34  both  during 
the  tropical  storm  and  the  two  sampling  days  that  followed  (Figure  A36).  In  clear  tropical 
waters  Kd  is  typically  0.05,  indicating  that  the  waters  in  Puerto  Rico  were  substantially  more 
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turbid  than  normal  during  the  entire  study  period  and  for  several  days  afterwards. 
Additionally,  wind  speed  was  near  the  upper  end  of  operational  limits  experienced  at  the 
Florida  fixed  sites  during  mosaic  sampling  (average  wind  speed  was  between  14- 
15knots)(Figure  A36).  While  these  wind  speeds  are  not  out  of  range  of  those  successfully 
surveyed  in  Florida  (Table  A8),  wind  gusts  during  the  study  period  were  above  25  knots  and 
during  the  previous  day  were  near  50  knots.  At  the  Media  Luna  site  the  cumulative  effect  of 
multiple  high  wind  days  caused  decreased  visibility  to  persist  even  after  average  wind  speeds 
fell  back  within  operational  ranges  (Figure  A36  see  IQ  data). 


Environmental  variables,  December  2007: 
Media  Luna  Reef,  Puerto  Rico 


Wind  gusts 


■  Avg  Wind  Speed 


- AvgKd 


Figure  A36.  Environmental  data  from  Media  Luna  Integrated  Coral  Observing  Network  (ICON)  station  for 
the  month  of  December  2007).  Average  and  Maximum  wind  speeds  are  shown  in  knots  while  visibility  is 
given  by  the  light  attenuation  coefficient  Kd.  December  11th,  the  date  in  which  Tropical  Storm  Olga  passed 
over  Puerto  Rico  is  highlighted  in  Red;  survey  dates  are  highlighted  in  blue. 


The  result  was  decreased  underwater  visibility  and  consequently  decreased  contrast  in  the 
acquired  images.  An  example  image  from  Media  Luna  Reef  collected  on  December  12th, 
2007  is  shown  below  (Figure  A37). 
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Figure  A37.  Image  of  benthos  from  ~2m  at  Media  Luna  Reef  on  December  12th,  2007. 

The  high  degree  of  sediment  suspended  in  the  water  column  created  a  very  low-contrast 
dataset  for  the  mosaic  creation  algorithms.  As  such,  only  small  portions  of  high  contrast 
images  from  the  study  sites  were  successfully  matched  and  assembled  into  a  composite 
mosaic  image.  It  should  also  be  noted  that  these  limited  results  were  only  available  after 
considerable  color  and  contrast  adjustments  to  the  original  video  frames  (Figure  A3  8). 
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Figure  A38.  Mosaic  of  Media  Luna  Reef  video  data  from  December  12th,  2007.  Only  a  handful  of  frames 
containing  high  contrast  objects  (such  as  the  large  Montastrea  colony  above)  were  successfully  matched  using 
this  dataset. 

The  low-contrast  of  the  majority  of  the  dataset  prevented  an  entire  landscape  video  mosaic  to 
be  created  with  this  dataset.  As  a  result  it  is  recommended  that  video  mosaic  data  only  be 
acquired  when  average  wind  speeds  are  less  than  1 5  knots  and  that  sufficient  time  be  given 
after  multiple  high  wind  days  to  allow  for  sediment  settlement  before  mosaic  acquisition  is 
attempted. 

Finally  it  is  recommended  that  while  it  is  possible  to  successfully  mosaic  video  imagery 
acquired  under  a  range  of  environmental  conditions,  the  best  products  with  the  highest 
benthic  resolution  are  acquired  under  the  best  visibility  conditions. 

2.  Camera  motion:  Guidelines  for  collecting  good  video  for  mosaic  creation  are  no  different 
than  those  required  for  collecting  good  photographs  underwater.  Blurry  images  with  a  lot  of 
motion  will  impede  the  mosaic  algorithm.  Camera  motion  can  be  controlled  conducting 
surveys  at  low  speed  (this  applies  to  divers,  ROVs,  and  AUVs)  and  avoiding  conditions  in 
which  high  surge  or  currents  cause  rapid  shifts  in  position  along  the  survey  track. 
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The  blur  caused  by  camera  motion  can  be  quantified  by  the  length  (in  pixels)  that  a  given 
image  point  moves  during  the  image  exposure  interval.  For  our  image  processing  algorithms 
it  is  not  advisable  to  have  motion  blur  more  than  4  pixels,  since  it  reduces  significantly  the 
ability  to  properly  register  the  images.  To  have  a  guideline  on  the  maximum  speed  allowed 
we  consider  our  typical  case  where  the  camera  is  straight  looking  down  at  the  bottom  at  a 
height  of  h  meters  and  translating  at  a  speed  of  v  meters  per  second.  The  camera  lenses 
provide  a  horizontal  field  of  view  of  a  degrees  and  the  image  sensor  has  r  pixels  along  the 
horizontal.  The  shutter  speed  is  5  second'1.  With  these  parameters  the  motion  blur  b  is 

v 

approximated  by  b  = - .  By  imposing  that  the  blur  should  be  smaller  than  bo= 4 

sin(a  /  r)-h-  s 

pixels,  we  can  establish  a  condition  for  the  maximum  allowed  translation  speed  for  a  given 
shutter  speed:  v  <  sin(a  /  r)  ■  h  •  b0  ■  s  .  In  practical  conditions  the  shutter  speed  should  be  set 

to  the  highest  possible  value  that  does  not  produce  a  noticeable  darkening  of  the  image. 
Therefore  it  should  be  adjusted  in  the  field  to  match  the  particular  illumination  conditions  of 
that  day.  Figure  A3  9  presents  the  plots  for  a  high  definition  video  camera  with  60  deg  field  of 
view,  for  different  heights  above  the  sea  floor. 
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Figure  A39  -Maximum  allowed  speed  of  translation  for  the  camera  as  a  function  of  the  shutter  speed  for 
different  altitudes.  The  plots  were  computed  for  a  high  definition  video  camera  with  60  deg  horizontal  field  of 
view,  and  motion  blur  under  4  pixels. 


In  addition,  camera  motion  should  be  minimized  by  not  making  any  sudden  turns  or 
movements  with  the  video  camera  during  acquisition.  This  is  accomplished  by  the  diver 
moving  around  the  camera  to  push  it  in  a  new  direction  while  swimming  the  lawn-mower 
pattern  over  the  area  of  interest  while  keeping  the  camera  in  the  same  orientation  throughout 
the  survey. 


3.  Motion  of  organisms  on  the  bottom:  The  same  issues  that  apply  to  camera  motion  apply  to 
the  motion  of  organisms  like  fish,  macroalgal  fronds,  and  soft  corals  that  can  translate  into 
blurry  images  that  can  cause  problems  for  the  mosaicing  algorithm.  To  minimize  these 
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impacts,  surveys  should  be  conducted  at  times  where  the  motion  of  organisms  on  the  bottom 
caused  by  surge  and  currents  is  minimized.  Due  to  the  natural  surge  of  areas  of  interest,  the 
mosaicing  algorithms  are  designed  to  find  matches  between  two  images  that  follow  the  same 
motion  pattern.  This  eliminates  some  of  the  problems  with  small  movements  of  individual 
soft  corals  or  fish  within  the  field  of  view  as  the  matching  algorithms  are  designed  to  follow 
the  features  of  the  more  static  benthos.  However,  this  feature  of  the  matching  algorithm  will 
not  work  if  the  moving  object  fills  the  entire  field  of  view.  Therefore,  it  is  beneficial  to  avoid 
large  areas  of  swaying  soft  corals  or  swimming  divers  while  acquiring  mosaic  data.  Soft 
corals  in  particular  will  also  often  block  the  view  of  other  benthic  organisms  and  hinder  the 
analysis  of  benthic  features.  Thus  areas  of  high  density  of  swaying  soft  corals  should  be 
avoided.  In  addition,  the  deployment  of  flexible  tapes  that  demark  areas  of  interest  can  sway 
with  the  currents  and  cause  problems  during  processing.  Markers  used  to  identify  areas  or 
organisms  of  interest  should  be  weighted  and  stationary  to  provide  minimal  artifacts  during 
processing.  Numbered  ceramic  tiles  are  heavy  enough  that  they  do  not  move  in  normal  surge 
and  present  a  large  surface  area  for  numbers  to  be  read  from  video,  making  them  ideal 
markers  for  mosaic  acquisition. 

4.  Lighting:  Imaging  of  benthic  features  is  best  completed  under  moderate  sunlight  conditions. 
While  light  is  needed  for  imaging  any  set  of  benthic  data,  high  sunlight  can  often  cause 
artifacts  during  mosaic  processing.  Even  lighting  has  been  found  to  create  the  best  mosaic 
products,  as  shown  in  Figure  A40.  In  this  figure  we  have  compared  two  sites  of  similar 
depth  (~2.5m)  under  different  light  conditions.  The  site  on  the  left  was  taken  under  overcast 
conditions  with  no  direct  sunlight  reaching  the  benthos,  while  the  example  on  the  right  was 
taken  on  a  sparsely  cloudy  day  with  ample  direct  sunlight.  While  sunflickering  effects  (as 
seen  in  the  right  image)  can  be  removed  using  the  module  developed  in  this  project  (See 
section  A2. 1.3.3),  it  is  still  recommended  that  video  footage  (if  possible)  be  collected  at  times 
or  conditions  when  lighting  of  the  benthos  is  even.  This  is  usually  best  accomplished  on 
overcast  days.  Deeper  sites  often  limit  light  penetration  and  thus  lighting  and  image  quality 
are  less  variable  during  the  day.  At  deeper  sites  color  loss  is  often  more  important,  and  if 
possible,  images  should  be  taken  with  a  red  fdter  at  depths  deeper  than  10m. 


Figure  A40.  Effect  of  light  variations  on  image  products.  Still  camera  image  from  Carysfort  reef  (~2.5m 
deep)  on  an  overcast  day  (Left).  Video  image  from  shallow  (~2.5m)  Acropora  palmata  site  in  St.  Croix  on  a 
sunny  day  (Right).  Light  refraction  against  the  bottom  is  clearly  seen  at  the  St.  Croix  site. 
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5.  Image  overlap:  The  mosaicing  algorithm  depends  on  a  minimum  of  60%  overlap  between 
images  to  process  effectively.  A  lack  of  sufficient  image  overlap  may  lead  to  failure  of  image 
registration  which  results  in  parts  of  the  survey  being  rejected  during  processing,  or  in  visible 
artifacts  in  the  mosaic  (such  as  the  same  benthic  objects  appearing  repeated  in  different 
locations).  Most  of  these  failures  can  be  compensated  by  manually  performing  the  image 
matching.  However  this  is  a  tedious  process  requiring  user  intervention. 

To  adequately  cover  the  area  of  interest  it  is  suggested  that  surveyors  use  a  double 
lawnmower’s  pattern.  This  pattern  comprises  two  simple  lawnmower’s  patterns  rotated  90 
degrees  between  each  other.  Figure  A41  illustrates  this  pattern.  The  double  lawnmower 
pattern  provides  extensive  overlap  among  perpendicular  strips.  To  increase  the  ability  of  the 
diver  to  follow  the  intended  coverage  pattern,  it  is  recommended  that  markers  are  deployed 
on  the  bottom  to  delimit  the  boundaries  of  the  area  and  that  these  landmarks  are  used  to 
ensure  that  each  survey  track  provides  adequate  overlap. 


Figure  A41  -  Example  of  a  area  coverage  pattern  comprising  two  lawnmower’s  patterns  rotated  90  degrees. 


6.  Image  processing:  As  discussed  above,  low  visibility  is  the  primary  limiting  factor  for  the 
collection  of  good  video  footage  to  be  used  for  mosaics.  As  a  rule  of  thumb,  mosaic  surveys 
should  be  avoided  when  sedimentation  or  water  color  impedes  a  clear  view  of  the  bottom 
(See  part  1  of  this  section).  In  cases  where  sediment  re-suspension  is  high,  the  video  camera 
and  the  mosaic  algorithm  may  focus  on  the  moving  sediment  particles  and  impede 
processing.  Due  to  the  fact  that  field  data  is  often  expensive  to  collect,  many  datasets  have 
been  collected  and  successfully  mosaiced  under  less-than  optimal  conditions.  As  such  a 
number  of  steps  have  been  put  in  place  to  deal  with  less  than  ideal  conditions  of  water  clarity. 
In  particular  pre-processing  of  video  data  using  standard  color  and  contrast  correction  often 
provides  much  clearer  images  than  were  possible  to  extract  in  the  field  on  the  day  of 
collection.  Several  sub-routines  have  been  developed  in  Matlab  to  do  batch  processing  of  the 
large  sets  of  video  data,  but  image  manipulation  software  such  as  Photoshop  are  also  capable 
of  the  same  results.  Below  is  an  example  of  mosaicing  results  before  and  after  color  and 
brightness  correction  (Figure  A42).  In  this  example,  raw  video,  taken  in  the  field,  was 
degraded  by  the  presence  of  sediment  suspended  in  the  water  column,  decreasing  visibility. 
When  attempting  to  mosaic  this  dataset  without  pre-processing  only  10  frames  were  able  to 
be  matched  (Figure  A42.A).  After  pre-processing  the  extracted  frames  using  color  and 
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contrast  correction  the  entire  set  of  images  was  able  to  be  assembled  into  a  single  mosaic 
image  (Figure  A42.B).  In  this  case,  the  sediment  and  decreased  water  clarity  prevented 
enough  benthic  texture  for  the  successful  use  of  the  landscape  mosaic  feature  matching 
algorithms  in  the  original  video  frames.  However,  simple  procedures  for  increasing  the 
benthic  clarity  were  able  to  overcome  these  limitations  and  allow  the  creation  of  a  mosaic 
image  of  higher  clarity  than  the  original  video  frames  (Figure  A42).  This  ability  to  pre- 
process  extracted  video  frames  has  allowed  the  landscape  mosaicing  software  to  be  used  in  a 
more  robust  set  of  working  environments  including  areas  where  the  visibility  was  less  than 
20  feet.  However  it  must  be  noted  that  optimum  sampling  will  occur  with  high  water  clarity 
and  visibility. 


Figure  A42.  Benefits  of  image  pre-processing.  Video  data  from  a  shallow  patch  reef  in  the  Florida  Keys  was 
hampered  by  low  visibility  and  suspended  sediments  in  the  water  column  during  data  collection. 
Unprocessed  video  frames  only  produced  matches  between  10  video  frames  from  the  dataset  (A)  while  color 
and  contrast  corrections  allowed  the  entire  dataset  to  be  used  and  mosaiced  despite  poor  initial  conditions 
(B). 

7.  Image  quality.  As  discussed  above,  many  factors  can  influence  the  successful  mosaicing 
of  an  image  dataset.  In  an  attempt  to  define  parameters  for  the  successful  use  of  the 
landscape  mosaicing  algorithms  we  compared  the  image  quality  of  several  datasets  collected 
during  the  course  of  this  project.  The  results  of  this  comparison  are  shown  in  Table  A9  and 
Table  A10. 


Table  A9.  Image  quality  characteristics  for  mosaic  datasets. 
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Dataset 

Avg  red 

std  dev 

Avg  green 

std  dev 

Avg  blue 

std  dev 

Histogram 

Example 

Frame 

Poor  Mosaic 
Data  (no  mosaic 
produced) 

E:\serdp\data\PuertoRico200A200712 
13 ICON\DV 

128.08 

21.37 

147.01 

19.71 

146.14 

19.48 

1 

M 

E:\serdp\data\Florida 
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I 

£ 
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Color 
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m 

Excellent  Mosaic 

Data 

E:\sGrdp\data\Florida2009\20090602_ 
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i 

£ 
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152.66 
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162.00 

35.01 

115.67 
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(21 

1 
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90.28 
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83.61 

40.47 

69.55 

37.03 

B 

Effects  of  Color 

Correction 

E:\Stills\Florida2009\20090624_HorsGS 
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40.02 
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k 
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hoG\colorll 
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28.18 

* 

S:\raw_Stills\Andros2009\20090314_Q 

6-3\127ND200 
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L 

m 

E  :\Sti  1  ls\And  ros2009\200903 15_Q6- 
3\color 

111.64 

49.99 

113.52 

54.74 

108.62 

57.31 

Kir 

In  this  analysis,  the  average  and  standard  deviation  of  the  digital  numbers  of  the  red,  green, 
and  blue  channels  were  calculated  for  a  subset  of  images  extracted  from  each  dataset.  In 
addition,  the  average  histograms  and  example  images  are  shown.  This  comparison  allows  us 
to  determine  the  range  of  image  quality  conditions  with  which  the  landscape  mosaic 
algorithms  are  capable  of  handling.  In  general,  the  two  datasets  that  did  not  allow  for  mosaic 
creation  (in  their  original  form)  were  characterized  by  relatively  narrow  sharp  peaks  (Table 
A9  grey  shaded  rows).  The  low  standard  deviation  of  digital  numbers  observed  in  each  color 
channel  of  these  datasets  indicates  very  low-contrast  images.  The  position  of  the  sharp  peaks 
to  the  left  of  the  image  also  indicates  a  preponderance  of  shadows  in  the  dataset.  The 
presence  of  a  single  narrow  peak  can  indicate  the  preponderance  of  a  single  color  in  the 
dataset  (Table  A9  blue  shaded  row).  While  this  can  create  a  skewed  result,  the  presence  of  a 
single  narrow  peak  may  not  preclude  successful  mosaic  creation  and  maybe  corrected  using 
color  and  contrast  adjustments.  The  presence  of  sharp  narrow  bands  were  absent  in  the 
datasets  that  were  considered  excellent  for  mosaic  creation  (Table  A9  orange  shaded  rows). 
These  datasets  are  characterized  by  very  wide  peaks  across  all  color  bands,  indicating  high 
contrast  color  images.  These  results  produced  the  best  mosaic  images  for  ecological 
analysis.  It  should  also  be  noted  that  the  use  of  color  and  contrast  correction  in  general 
created  images  that  had  wider  histogram  bands  after  processing  (Table  A9;  the  green  shaded 
rows  show  the  pre-processed  and  processed  data).  Finally,  an  examination  of  the  datasets 
collected  from  the  permanent  sites  Brooke’s  Reef  and  Grecian  Rocks  showed  a  wide  variety 
of  image  quality  examples  that  were  successfully  matched  using  the  landscape  mosaic 
algorithms  (Table  A 10). 

Table  A10.  Image  quality  information  for  mosaic  data  acquired  at  permanent  monitoring  sites  Brooke’s  Reef 

and  Grecian  Rocks. 
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Dataset 

Avg  red 

std  dev 

Avg  green 

std  dev 

Avg  blue 

std  dev 

Histogram 

Example 

Frame 
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While  these  datasets  were  mosaiced  under  a  wide  range  of  image  quality  conditions,  in 
general  bands  were  wide  and  shallow.  This  range  of  successful  processing  parameters 
indicates  that  it  is  the  presence  of  sharp  narrow  peaks  among  multiple  channels  that  prohibits 
successful  mosaicing  (Table  A10).  The  average  standard  deviation  in  each  band  of  the 
mosaics  that  were  not  successfully  processed  was  between  13.7  and  22.5,  whereas  the 
average  standard  deviation  of  the  mosaics  that  were  either  assessed  as  excellent  or  acceptable 
ranged  from  35.4  to  46.7.  The  color-skewed  mosaic  from  the  deep  site  in  the  Dry  Tortugas 
had  one  channel  (red)  with  low  contrast  (S.D.  =  24.1)  but  two  channels  with  high  contrast 
(green  and  blue  with  S.D.  =  39.7  and  40.7,  respectively).  From  these  data,  we  can  suggest  a 
rule  of  thumb  that  images  with  standard  deviations  in  all  bands  less  than  about  20  digital 
numbers  will  not  produce  acceptable  mosaics  (Table  A1 1). 


Table  All:  Average  standard  deviations  for  mosaics  that  could  not  be  created  (“poor  mosaics”)  compared 
with  mosaics  that  were  successfully  created  (the  other  rows).  Data  taken  from  Tables  A9  and  A10. 


Dataset 

Standard  deviation 
in  red  band 

Standard  deviation 
in  green  band 

Standard  deviation 
in  blue  band 

Poor  mosaics 

22.5 

21.6 

13.7 

Color  skewed 

24.1 

39.7 

40.7 

Excellent  mosaics 

37.7 

39.6 

35.4 

Mosiacs  from  FL 

41.5 

46.7 

39.7 
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A2.4  Applications  of  coral  reef  monitoring  and  mapping  using  landscape 
mosaic  technology 

A2.4.1  Ship  Grounding 

Vessel  groundings  are  a  major  source  of  disturbance  to  coral  reefs  worldwide.  Documenting  the 
extent  of  damage  caused  by  groundings  is  a  crucial  first  step  in  the  reef  restoration  process. 
Video  mosaics,  created  by  merging  thousands  of  video  frames,  combine  quantitative  and 
qualitative  aspects  of  damage  assessment  and  provide  a  geo-referenced,  landscape,  high- 
resolution,  spatially  accurate  permanent  record  of  an  injury. 

The  active  rehabilitation  and  restoration  of  damaged  reef  habitats  in  the  US  relies  largely  on  the 
ability  of  authorities  with  jurisdiction  over  the  resources  to  prosecute  the  parties  responsible  for 
the  damage  and  retain  monetary  recoveries  that  can  be  used  directly  for  restoration  (Precht  and 
Robbart  2006;  Shutler  et  al.  2006).  To  determine  the  proper  amount  of  restoration  required,  a 
two-stage  Natural  Resource  Damage  Assessment  (NRDA)  is  conducted  to  determine:  (1)  the 
“primary”  actions  needed  to  return  the  habitat  to  its  original  baseline  structure  and  function;  and 
(2)  the  “compensatory”  actions  needed  to  compensate  the  public  for  the  loss  of  resources  and 
services  until  primary  restoration  is  completed  (Symons  et  al.  2006).  Central  to  the  NRDA 
process  is  the  accurate  and  comprehensive  quantification  of  the  damage  caused  by  a  vessel  on  a 
benthic  community.  In  this  study,  we  describe  the  application  of  a  novel  methodology,  landscape 
video  mosaics  that  are  ideally  suited  for  the  quantification  of  damage  caused  by  vessel 
groundings  on  coral  reefs  as  well  as  subsequent  recovery  patterns.  This  methodology  can,  with 
limited  time  in  the  field,  satisfy  the  crucial  initial  damage  assessment  needs  that  are  required  for 
the  subsequent  recovery  of  funds  from  responsible  parties  as  well  as  establish  a  visual  baseline 
of  the  damage  against  which  future  recovery  can  be  ascertained. 

On  5  December  2002,  the  49-foot  vessel  “Evening  Star”  ran  aground  on  a  hardbottom 
community  dominated  by  stony  and  soft  corals  within  the  waters  of  Biscayne  National  Park, 
Florida  (25°  23.332’  N,  80°  09.874’  W,  3  m  of  depth).  On  May  23,  2005  and  again  on  July  19, 
2006,  video  data  of  the  damaged  and  surrounding  areas  was  collected  using  a  Sony  TRV900  DV 
camcorder  placed  in  an  underwater  housing  following  the  methods  described  by  Lirman  et  al. 
(2007).  The  camera  operator  swam  a  lawnmower's  pattern  of  side-by-side  strips  followed  by  a 
similar  pattern  rotated  90°.  A  bubble  level  taped  to  the  back  of  the  camera  housing  helped  the 
diver  keep  the  camera  pointed  in  a  nominally  nadir  angle.  The  camera  operator  used  a  digital 
depth  gauge  to  keep  a  consistent  depth  during  the  surveys.  The  time  required  for  a  single  diver  to 
collect  the  video  used  for  mosaic  creation  was  less  than  one  hour  in  both  years. 

During  the  2006  survey,  positional  (GPS)  information  was  obtained  for  the  outline  of  the  injury 
as  well  as  25  ground-control  points  (GCPs)  along  the  periphery  of  the  scar  using  the  diver 
platform  of  the  Shallow  Water  Positioning  System  (SWaPS)  (Figure  A43.D).  The  GPS  tracks 
recorded  by  the  diver  were  used  to  demark  the  perimeter  of  the  scar,  and  the  area  of  the  scar  was 
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then  computed  from  the  polygon  delimited  by  the  scar  perimeter  using  linear  distances  between 
GCPs.  Positions  of  the  25  GCPs,  identified  using  numbered  ceramic  tiles  and  painted  disks  easily 
visible  in  the  video  (Figure  A43.C),  were  captured  by  the  SWaPS  platform  and  used  for  mosaic 
creation.  The  deployment  of  the  tiles  used  to  establish  the  position  of  the  GCPs  as  well  as  the 
SWaPS  survey  took  a  single  operator  less  than  one  hour. 


Figure  A43.  A)  Superficial  damage  to  a  coral  colony  caused  by  a  small  ship  grounding.  B)  Severe  reef 
framework  damage  caused  by  a  large  vessel  grounding.  C)  Numbered  tiles  and  painted  disks  used  as  Ground 
Control  Points  (GCPs)  for  mosaic  creation.  D)  Diver  version  of  the  Shallow  Water  Positioning  System 
(SWaPS)  used  to  determine  the  location  of  GCPs  along  the  scar  boundary.  The  unit  integrates  a  GPS  unit 
and  a  video  camera  to  provide  get-tagged  frames  of  the  bottom. 


The  scar,  imaged  in  2005  and  2006  in  a  Florida  reef  impacted  by  a  49-foot  vessel,  covered  an 
area  of  150  m2  (total  imaged  area  was  >  600  m2).  The  impacted  coral  community  showed  limited 
signs  of  coral  recovery  more  than  three  years  after  the  initial  impact;  the  cover  of  corals  was  still 
significantly  higher  in  the  undamaged  areas  compared  to  the  scar.  However,  seagrass 
colonization  of  the  scar  was  observed. 

Accurately  documenting  patterns  of  physical  damage  (and  subsequent  recovery  patterns)  to 
benthic  habitats  can  be  especially  challenging  when  the  spatial  extent  of  injuries  exceeds  tens  of 
square  meters.  These  large  injuries  are  often  too  difficult  to  measure  in  situ  by  divers  and  too 
small  or  costly  to  be  quantified  effectively  using  aerial  and  satellite  remote  sensing  tools. 
Moreover,  in  cases  where  immediate  action  is  required  to  initiate  recovery  efforts  and  avoid 
secondary  damage  to  the  resources,  damage  assessment  activities  need  to  be  conducted  quickly. 
Landscape  mosaics  capture  data  at  a  scale  between  diver  observations  and  aerial  imagery, 
thereby  providing  an  ideal  approach  to  assess  grounding  injuries  because:  (1)  the  images  are 
recorded  close  to  the  seabed  (<  2  m  from  the  bottom)  thus  capturing  detailed  visual  information, 
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(2)  the  resulting  mosaics  cover  large  areas  of  the  bottom  at  scales  commensurate  with  the 
damage  caused  by  large  vessel  groundings;  and  (3)  the  imagery  needed  to  document  patterns  can 
be  collected  quickly  with  an  underwater  video  camera  and,  optionally,  a  surface  GPS.  Using  this 
mosaic-based  (or  image-based)  methodology,  the  dimensions  of  the  injury  caused  by  the  49-foot 
cabin  cruiser  “Evening  Star”  in  December  2002  in  the  waters  of  Biscayne  National  Park,  Florida, 
as  well  as  the  condition  of  the  affected  benthic  community  were  documented  in  2005. 

In  addition  to  providing  a  method  for  measuring  the  extent  of  injuries,  landscape  mosaics  create 
a  spatially  accurate  map  of  the  distribution  and  condition  of  benthic  organisms  so  that  patterns  of 
recovery  (or  further  damage)  can  be  more  easily  assessed  than  by  diver-based  methods  alone. 
Repeat  mosaics  taken  over  time  at  the  same  location  can  be  used  to  measure  changes  to  a  study 
site  without  requiring  extensive  tagging  of  individual  organisms.  In  the  present  study,  a  second 
mosaic  of  the  same  grounding  scar  was  constructed  in  2006  to  assess  patterns  of  community 
succession  and  further  damage  caused  by  the  passage  of  four  hurricanes  (Dennis,  Katrina,  Rita, 
and  Wilma)  during  the  summer  of  2005  (Manzello  et  al.  2007). 

The  image-plus-GCP  mosaicing  method  differs  from  the  image-only  method  in  the  global 
optimization  step  (Figure  A44).  Under  the  image-only  method,  the  cost  function  that  is 
minimized  uses  only  the  image-to-image  registration  points  (Gracias  et  al.  2003).  In  contrast, 
under  the  image-plus-GCP  method,  the  cost  function  to  be  minimized  uses  terms  for  both  the 
image-to-image  registration  points  and  the  image-to-GPS  registration  points  (Farrer  et  al.  2007). 
In  both  the  image-only  and  image-plus-GCP  algorithms,  the  image  registration  process  estimates 
the  3D  position  and  orientation  of  the  camera  for  each  image  thus  accommodating  for  changes  in 
altitude  and  pitch  and  roll.  In  addition,  the  image-plus-GCP  algorithm  georeferences  the  mosaic 
to  a  world  coordinate  system  (Universal  Transverse  Mercator  Zone  17N  in  this  case).  Therefore, 
following  the  blending  step,  the  mosaics  created  with  GPS  input  are  directly  exportable  to  GIS 
software  or  Google  Earth  (Geotiff®  and  KMZ  formats). 
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B)  Imoge-Plus-GCP  Method 


Figure  A44.  Flow-charts  depicting  the  differences  between  the  Image-only  method  (A)  and  the  Image-Plus 
GCP  method  (B). 

The  use  of  mosaics  to  survey  the  damage  caused  by  groundings  shifts  the  bulk  of  time  needed  to 
complete  a  diver-based  classic  damage  assessment  from  the  field  to  the  lab.  The  time  required  to 
collect  both  the  video  (<  1  hour)  and  the  GCPs  (<  1  hour)  in  the  field  was  minimal  and  was 
easily  achieved  with  one  pair  of  divers.  The  processing  time  for  the  completion  of  the  landscape 
mosaics  ranged  from  5-10  days.  However,  it  is  important  to  note  that  most  of  the  processing 
steps  are  automated  and  therefore  require  only  minimal  operator  input,  so  the  actual  operator 
time  required  was  only  a  few  hours  for  each  mosaic.  More  importantly,  significant  improvements 
to  the  mosaic  algorithms  have  been  made  over  the  past  three  years,  and  total  processing  times  for 
mosaics  similar  to  those  presented  here  are  now  1-2  days.  The  processing  time  is  roughly  divided 
into  the  following  portions:  sunflickering  removal  (33  %  of  the  time),  global  matching  (64  %), 
optimization  (1%),  blending  (2%).  For  the  2006  mosaic,  documenting  the  position  of  the  GCPs 
from  the  geo-tagged  video  took  approximately  3  hours. 


A2.4.1.1  Status  and  Trends  of  the  Benthic  Community 

The  mosaics  of  the  grounding  site  created  in  2005  and  2006  were  analyzed  to  assess  the  percent 
cover  of  benthic  organisms  in  both  the  grounding  scar  and  adjacent,  undamaged  areas  as  well  as 
the  patterns  of  permanence  and  removal  of  stony  corals  between  2005  and  2006. 


72 


RC-1333  Final  Report  to  SERDP  2010 


R.P.Reid 


Figure  A45.  Landscape  video  mosaic  surveys  of  the  ship-grounding  damage  caused  by  the  Evening  Star  in 
2005  and  2006. 

All  information  on  the  percent  cover  of  the  dominant  benthic  taxa  (i.e.,  stony  corals,  soft  corals, 
sponges,  seagrass)  was  measured  using  the  point-intercept  method  within  replicate  1  m  x  1  m 
sections  of  the  mosaics  (Lirman  et  al.  2007).  For  this  assessment,  a  set  of  random  points  was 
superimposed  onto  each  mosaic  using  the  image  analysis  software  CPCe  (Kohler  and  Gill  2006). 
The  random  points  were  used  as  the  central  locations  of  simulated  1  m  x  1  m  quadrats  used  as 
sampling  units.  Once  the  quadrats  had  been  positioned  within  each  mosaic,  a  set  of  25  quadrats 
within  the  grounding  scar  and  25  quadrats  from  the  adjacent,  undamaged  areas  were  selected  (the 
first  25  random  points  were  selected  for  each  habitat  type).  Each  individual  quadrat  was  then 
analyzed  by  superimposing  25  random  points  within  its  boundary  using  CPCe  (Kohler  and  Gill 
2006).  The  identity  of  the  organism  or  bottom  type  immediately  under  each  point  was 
determined  and  the  percent  cover  of  each  category  was  calculated  as  the  proportion  of  the  points 
occupied  by  a  given  taxon  over  the  total  number  of  points  (i.e.,  25  points  per  quadrat)  as 
described  by  Lirman  et  al.  (2007).  The  percent  cover  data  were  analyzed  in  a  2-way  ANOVA 
with  time  (2005,  2006)  and  habitat  type  (scar,  undamaged  area)  as  factors.  Finally,  a  subset  of 
organisms  (corals,  sponges)  visible  in  the  2005  mosaic  were  identified  and  relocated  in  the  2006 
mosaic  to  determine  permanence  or  removal/mortality  between  surveys.  Because  mosaics  of  the 
same  area  collected  over  time  are  easily  referenced  to  each  other,  it  is  possible  to  determine  the 
location  of  organisms  or  features  in  different  surveys  without  the  need  to  deploy  markers.  Thus, 
removal  of  organisms  can  be  easily  determined  by  locating  their  initial  position  in  a  prior  survey. 

In  the  area  of  overlap  between  the  2005  and  2006  mosaics,  a  total  of  69  coral  colonies  were 
identified  in  2005.  Of  the  69  colonies  identified  in  2005,  62  (90%)  were  relocated  in  the  2006 
mosaic  showing  that  limited  physical  damage  was  experienced  by  this  site  due  to  the  2005 
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hurricanes  (or  any  other  potential  source  of  physical  damage  like  swells  or  additional 
groundings).  A  total  of  seven  colonies  were  removed  or  died  completely  between  surveys  (10 
%).  Lastly,  two  surviving  colonies,  both  within  the  scar,  appeared  to  have  become  dislodged  and 
moved  from  their  original  location  between  surveys. 

Three  years  after  the  initial  vessel  impact,  the  benthic  communities  in  damaged  and  undamaged 
areas  were  still  significantly  different  in  the  percent  cover  of  the  dominant  taxa.  The  cover  of 
stony  corals  and  soft  corals  was  significantly  higher  in  the  undamaged  areas,  while  no  significant 
differences  in  the  cover  of  sponges  between  habitats  were  detected.  No  significant  differences  in 
the  cover  of  corals  and  sponges  were  documented  between  2005  and  2006.  The  most  striking 
feature  of  the  inter-annual  comparison  was  the  evident  encroachment  of  seagrass  ( Thalassia 
testudinum )  into  the  scar  from  the  surrounding,  unaffected  habitat  (Figure  A46).  The  cover  of 
seagrass  within  the  scar  increased  significantly  between  2005  (3.7  %,  S.D.  =  4.4)  and  2006  (8.2 
%,  S.D.  =  8.5). 


Figure  A46.  Encroachment  of  seagrass  into  the  affected  area  between  the  2005  survey  (A)  and  the  2006 
survey  (B)  as  assessed  through  repeat  video-mosaics. 
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A2.4.1.2  Comments  and  Recommendations 

Providing  a  rapid  and  accurate  assessment  of  the  damage  caused  by  vessel  groundings  on  coral 
reef  habitats  is  a  crucial  first  step  in  the  reef  restoration  process.  This  new  application  of 
underwater  mosaics  covered  a  larger  area  (>  600  m2)  than  previous  surveys  with  this  technology 
and  demonstrated  the  potential  to  incorporate  external  navigation  into  the  mosaic  processing, 
thereby  enhancing  the  spatial  accuracy  of  the  resulting  landscape  map.  The  video  mosaics 
provided  a  means  to  accurately  and  efficiently  collect  information  on  the  size  of  the  damage  area 
as  well  as  the  status  and  trends  of  the  impacted  biological  communities,  thereby  expanding  the 
quality  and  diversity  of  information  that  can  be  collected  during  field  surveys.  The  damaged 
portion  of  the  reef  surveyed  in  Florida  covered  an  area  of  150  m2  and  the  impacted  coral  reef 
community  showed  limited  convergence  to  the  undisturbed  community  in  the  same  habitat  more 
than  three  years  after  the  initial  impact. 

For  a  complete  assessment  of  the  use  of  landscape  mosaic  technology  for  the  documentation  and 
monitoring  of  ship-grounding  disturbance  on  coral  reefs  please  see: 

Lirman,  D.,  N.  Gracias,  B.  Gintert,  A.  C.  R.  Gleason,  G.  Deangelo,  M.  Gonzalez,  E.  Martinez,  R. 
P.  Reid  (in  press):  Damage  and  Recovery  Assessment  of  Vessel  Grounding  Injuries  on  Coral 
Reef  Habitats  Using  Georeferenced  Landscape  Video  Mosaics.  Limnology  and  Oceanography: 
Methods. 


A2.4.2  Spatio-temporal  analysis  of  coral  communities 

As  part  of  an  ongoing  collaboration  with  the  coral  reef  monitoring  program  at  the  Atlantic 
Undersea  Test  and  Evaluation  Center  (AUTEC)  changes  in  the  abundance  and  spatial 
arrangement  of  benthic  organisms  of  an  inshore  patch  reef  at  Andros  Island,  Bahamas  were 
examined  over  a  35-year  time  period. 

Recent  decades  have  been  marked  by  extreme  changes  in  coral  reef  communities  both  in  terms 
of  abundance  and  community  composition.  Despite  extensive  research  into  the  causes  and 
effects  of  recent  changes  in  coral  communities,  little  is  known  about  how  the  spatial  distributions 
of  benthic  organisms  within  reefs  have  changed  over  recent  decades.  In  association  with  the 
coral  reef  monitoring  program  at  the  U.S.  Navy’s  Atlantic  Undersea  Test  and  Evaluation  Center 
(AUTEC)  at  Andros  Island,  Bahamas,  the  spatial  arrangement  of  benthic  organisms  within  a 
lOxlO-m  reef  plot  has  been  mapped  over  a  35  year  period,  providing  a  unique  historical  dataset 
to  evaluate  long-term  changes  in  abundance  and  spatial  distribution  of  reef  organisms. 

The  hand  drawn  coral  species  distribution  map  used  in  this  collaboration  consists  of  a  10m  xlOm 
area  of  an  inshore  patch  reef  just  offshore  of  the  Atlantic  and  Undersea  Test  and  Evaluation 
Center  (AUTEC)  (Figure  A47).  This  10x10m  reef  plot  was  installed  in  the  late  1960’s  for  use  in 
a  Navy  coral  reef  monitoring  program.  Within  the  1  Ox  10m  reef  plot  all  sessile  benthic 
invertebrates  were  surveyed  and  mapped  to  species  level  for  corals  and  genus  for  sponges, 
gorgonians  and  zoanthids. 
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Figure  A47.  Example  of  digitized  hand-drawn  species  distribution  map  of  a  10x10m  reef  patch.  The  area  has 
been  digitized  by  coral  species  with  the  following  color  codes:  orange  (Porites  porites),  purple  (Montastrea 
annularis  complex),  gray  (Montastrea  cavernosa),  red  (Agaricia  agaricites),  green  (Porites  astreoides). 

A  second  survey  of  the  inshore  patch  reef  site  was  made  in  February  2008  using  second- 
generation  mosaic  survey  technology  (Figure  A48).  The  resulting  video  mosaic  was  used  to 
document  the  location  and  areal  extent  of  coral  species  within  the  study  area. 

Both  the  hand  drawn  and  mosaic  derived  species  distribution  maps  were  georeferenced  using 
ArcGISv9.0.  The  boundaries  of  individual  coral  colonies  were  digitized  by  hand  and  categorized 
by  species.  Analysis  of  spatial  distribution  patterns  of  individual  species  was  accomplished  by 
examining  the  nearest  neighbor  statistics,  spatial  means  and  standard  deviational  ellipses,  and  a 
test  for  local  spatial  autocorrelation  (the  Getis-Ord  G,*). 
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Figure  A48.  Partially  digitized  high  resolution  video  mosaic  of  the  lOxlOm  plot  located  at  site  Sl-10.  Coral 
species  are  coded  by  color:  Orange  (Porites  porites),  Purple  (Montastrea  annularis  complex),  Green  (Porites 
astreoides),  and  Light  Blue  (Montastrea  cavernosa). 

Live  coral  cover  within  the  lOmxlOm  study  plot  showed  a  slight  increase  over  the  35  year  time 
period  (14.0%-15.7%).  Despite  this  increase  the  number  of  colonies  found  in  the  study  plot 
declined  significantly  (from  350  to  260)  while  the  average  colony  size  was  found  to  increase 
over  the  time  period  (from  3.8cm2  to  6.0cm2).  These  changes  indicate  not  only  a  loss  of  small 
colonies  but  also  significant  growth  of  large  colonies  resulting  in  the  slightly  higher  coral  cover. 

Examination  of  the  spatial  distribution  maps  of  the  two  time  periods  (Figure  A49.A)  indicates 
significant  changes  in  the  arrangement  of  coral  colonies  within  the  site  despite  the  similar  levels 
of  abundance  at  the  two  time  periods.  For  comparison  purposes  the  digitized  area  coverages  of 
the  two  distributions  were  gridded  and  the  1972  distribution  was  subtracted  from  the  2008  areal 
distribution  map.  Changes  in  coral  cover  are  shown  in  figure  A49.B.  Areas  of  high  mortality  are 
shown  in  red  and  areas  of  growth  are  depicted  in  dark  blue. 
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Figure  A49.  A)  The  1972  (purple)  and  2008  (orange)  distributions  of  live  coral  cover  at  the  study  site.  The 
grid  shows  the  V*  m  grid  cells  used  to  create  the  change  in  area  map  (B).  B)  An  interpolated  area  map  based 
on  the  change  in  live  coral  cover  values  from  1972-2008.  Areas  in  red  represent  areas  of  high  coral  cover  loss 
within  grid  cells  (-2500  cm2-  -1000  cm2)  orange  represents  moderate  loss  (-1000  cm2  -230  cm2)  green  is  little 
to  no  loss  (-230  cm2  -350  cm2)  light  blue  is  moderate  growth  (350  cm2  -1000  cm2)  and  dark  blue  is  high 
growth  (1000  cm2  -2500  cm2). 

In  order  to  determine  if  local  areas  within  the  1  Ox  10m  plot  were  prone  to  coral  mortality  or 
growth  (as  opposed  to  a  random  pattern)  a  clustering  analysis  was  performed  on  the  change  in 
area  information.  The  results  were  mapped  over  the  original  area  (Figure  A50.B) 


Figure  A50.  A).  Shows  the  1972  and  2008  distributions  of  live  coral  cover  within  the  study  area.  B)  Depicts 
the  results  of  the  local  indicators  of  spatial  autocorrelation  Getis-ORD  G  test  over  the  same  study  area  as  seen 
in  (A).  Significant  negative  changes  in  area  (high  mortality)  are  shown  in  red.  Significant  areas  of  high 
positive  changes  in  area  (high  growth)  are  shown  in  dark  blue. 
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The  local  indicators  of  spatial  autocorrelation  test  (Getis-Ord  G,  )  indicate  that  neither  mortality 
nor  growth  during  the  35  year  time  period  occurred  in  a  random  pattern.  The  top  left  of  the 
quadrat  seems  to  have  undergone  significant  mortality  while  growth  was  localized  in  the  lower 
right  portion  of  the  study  area. 

One  of  the  most  promising  features  of  the  spatial  analysis  of  coral  communities  is  that  by 
mapping  clusters  of  high  mortality  and  high  growth  characteristics  over  time  we  may  be  able  to 
identify  local  environmental  or  habitat  characteristics  that  maybe  influential  in  the  persistence  of 
coral  communities  in  these  locations.  From  the  present  study  we  found  that  the  finger  coral 
Porites  porites  had  the  highest  change  in  spatial  distribution  over  time  at  the  study  site.  These 
analyses  also  showed  distinct  regions  of  high  growth  and  mortality  characteristics  for  this  species 
over  the  3  5 -year  time  period.  As  an  initial  attempt  to  understand  these  patterns  we  overlaid  the 
two  distributions  over  a  depth  map  of  the  area  (Figure  A5 1). 


Figure  A51.  The  1972  (blue)  and  2008(purple)  distributions  of  the  coral  species  Porites  porites  overlaid  on  top 
of  a  depth  map  of  the  study  area.  Graph  shows  the  %  live  coral  cover  of  P.  porites  by  depth  for  1972  (green) 
and  2008(purple). 

From  this  it  was  apparent  that  the  majority  of  the  2008  distribution  was  found  in  deeper  water 
than  the  1972  distribution.  In  addition  the  only  areas  of  overlap  between  the  two  distributions 
(found  in  the  lower  right  portion  of  the  study  area)  are  again  located  in  slightly  deeper  waters 
than  the  majority  of  the  1972  distribution.  The  smaller  colonies  located  in  2008  (and  those  most 
likely  newest)  are  located  in  deeper  portions  of  the  reef  than  the  rest  of  the  2008  distribution. 
Although  not  conclusive  these  findings  suggest  that  depth,  or  other  environmental  variables 
related  to  depth  (light  intensity,  temperature  etc)  may  have  played  a  role  in  the  survivorship  and 
persistence  of  Porites  porites  over  the  last  35  years  at  this  site. 

The  results  of  the  spatial  analysis  represent  some  of  the  first  analyses  of  changes  in  spatial 
distributions  over  time  for  coral  species.  Although  limited  in  its  spatial  extent  these  results  show 
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that  despite  being  long-lived  species,  corals  do  show  changes  in  their  distributions  over  time. 
Additionally  the  use  of  local  cluster  analyses  such  as  the  Getis-ORD  G;  have  provided  coral 
ecologists  with  a  new  tool  in  which  to  view  and  map  local  areas  of  mortality  and  growth  that  can 
provide  more  information  on  coral  community  dynamics  than  changes  in  abundance  alone.  This 
is  especially  apparent  at  the  study  site  discussed  here  in  which  measures  of  abundance  stayed 
relatively  constant  over  the  35  year  time  span  while  the  distributions  of  corals  within  this  area 
changed  significantly.  In  addition  the  ability  to  create  local  maps  of  high  mortality  and  high 
growth  characteristics  may  allow  researchers  to  identify  local  environmental  or  habitat 
characteristics  that  maybe  beneficial  to  the  persistence  of  coral  reef  communities  into  the  future. 


A2.4.3  Monitoring  and  Assessment  of  Mesophotic  coral  communities 

Landscape  mosaic  products  provide  an  excellent  record  of  the  state-of-the-reef  at  the  time  of  a 
survey  and  can  be  used  over  time  to  monitor  changes  in  reef  communities.  Because  of  the  short 
dive  times  required  for  acquisition,  these  tools  are  uniquely  suited  for  use  in  reef  environments 
that  are  not  easily  accessible.  As  such  we  have  collaborated  with  other  researchers  at  the 
University  of  Miami  and  at  the  University  of  the  Virgin  Islands  to  survey  and  create  baseline 
data  for  mesophotic  reef  communities. 

Well-developed  coral  communities  in  mesophotic  habitats  have  been  historically  under¬ 
represented  in  regional  reef  monitoring  programs  due  to  difficulties  associated  with  access  of 
trained  observers  to  deep  areas.  Mesophotic  coral  reef  communities  may  play  a  significant  role 
as  essential  habitat  for  associated  reef  organisms  and  as  potential  refugia  (Riegl  and  Piller  2003) 
from  which  shallow-water  coral  ecosystems  might  recover  from  disturbance  events  that  have 
decimated  shallow-water  counterparts  (Gardner  et  al.  2003).  In  this  study,  we  describe  landscape 
video  mosaics,  which  have  several  advantages  for  monitoring  the  status  and  trends  of  mesophotic 
reef  habitats.  Landscape  mosaics  can  help  circumvent  the  limitations  to  diver-based  monitoring 
imposed  by  limited  bottom  time  at  mesophotic  depths  because  they  consist  of  high-resolution, 
spatially  accurate  imagery  that  can  be  analyzed  to  extract  key  metrics  of  coral  community 
condition  such  as  cover,  prevalence  of  disease  and  bleaching,  and  colony  sizes. 

The  effectiveness  of  using  landscape  mosaics  for  mapping  and  monitoring  shallow  reefs, 
suggests  that  landscape  mosaics  may  also  prove  useful  for  mosaicing  mesophotic  coral 
ecosystems.  The  goals  for  this  study  were  to  mosaic  video  data  from  two  regions  containing 
extensive  coral  ecosystems  at  mesophotic  depths  (the  US  Virgin  Islands  and  the  Dry  Tortugas, 
USA),  analyze  the  mosaics  to  measure  the  cover  of  dominant  benthic  taxa,  and  determine  if 
cover  from  mosaics  was  comparable  to  diver-based  measurements  for  the  same  areas. 

As  part  of  the  collaborative  effort  between  the  University  of  Miami  and  UVI,  two  mesophotic 
reef  sites  were  surveyed  and  analyzed.  Site  one  (24°  42.295’  N,  83°  2.622’  W,  23  m  depth)  in 
the  Sherwood  Forest  region  of  the  Dry  Tortugas,  FL  was  surveyed  on  17  May  2005.  This  general 
area  has  been  described  by  Miller  et  al.  (2001).  Video  data  were  acquired  with  a  Sony  TRV900 
digital  video  camera  in  an  Aquavideo  housing  using  ambient  lighting.  The  comers  of  an 
approximately  7  x  7  m  survey  plot  were  marked  with  inflatable  diver  signaling  devices  to  help 
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guide  the  diver  while  swimming,  and  three  25  x  25  cm  PVC  quadrats  were  placed  within  the 
survey  plot  for  scale. 

Site  two  (18°  11.8083’  N,  65°  04.7451’  W,  38.5  m  depth)  in  the  Hind  Bank  Marine  Conservation 
District  (MCD),  US  Virgin  Islands  (USVI)  was  surveyed  on  4  December  2007.  This  area  has 
been  described  by  Smith  et  al.  (2009).  Video  data  were  acquired  with  a  Sony  TRV900  digital 
video  camera  in  a  Light  and  Motion  Stingray  II  housing.  A  Halcyon  Apollo  light  system  with 
dual  50  W  HID  lamps  was  used  to  provide  supplementary  artificial  illumination.  The  comers  of 
an  approximately  5  x  5  m  survey  plot  were  marked  with  Styrofoam  floats  and  a  single  25  x  25 
cm  PVC  quadrat  was  placed  within  the  survey  plot  for  scale. 

At  both  sites,  a  single  diver  swam  transects  over  the  survey  plot  while  holding  the  video  camera 
as  vertical  as  possible.  One  set  of  parallel  transects  was  completed  with  high  (at  least  50%) 
overlap  between  the  swaths  and  then  a  second  set  of  transects  with  little  overlap  between  the 
swaths  was  acquired  perpendicular  to  the  first  set. 

Landscape  mosaics  of  both  sites  were  created  using  the  procedure  described  by  Lirman  et  al. 
(2007).  The  cover  and  composition  of  the  coral  communities  on  the  two  survey  plots  were 
assessed  directly  from  each  mosaic  using  the  point-intercept  method  (Lirman  et  al.  2007).  Four 
hundred  random  points  were  superimposed  onto  each  image  using  the  CPCe  software  (Kohler 
and  Gill  2006)  and  the  identity  of  the  organisms  or  bottom  type  found  directly  under  each  point 
was  determined. 

In  the  Dry  Tortugas,  in  addition  to  collecting  the  video  for  the  construction  of  the  landscape 
mosaics,  divers  estimated  the  percent  cover  of  stony  corals  along  four  10  m  line-intercept 
transects  following  the  Atlantic  and  Gulf  Rapid  Reef  Assessment  (AGRRA)  protocol  (Kramer 
and  Lang  2003).  One  of  these  four  transects  was  at  the  site  of  the  mosaic,  the  other  three  were 
within  500  m  of  the  mosaic.  Additional  data  at  the  USVI  site  included  habitat  type,  benthic 
composition,  and  coral  health  (Smith  et  al.  2009).  Benthic  cover  was  extracted  from  four  20  -  30 
m  video  transects,  producing  63  to  87  non-overlapping  frames.  Percent  cover  of  benthic 
categories  was  determined  under  10  randomly  placed  points  per  frame  (Smith  et  al.  2008).  One 
of  these  four  transects  was  at  the  site  of  the  mosaic,  other  three  were  within  1  km  of  the  mosaic. 

The  Dry  Tortugas  mosaic  (Figure  A52)  covered  52  m2  with  an  approximate  spatial  resolution  of 
2.3  mm/pixel.  The  USVI  mosaic  (Figure  A53)  covered  31  m2  with  an  approximate  spatial 
resolution  of  1.8  mm/pixel.  Cover  of  stony  coral  was  high  at  both  sites  but  coral  cover  at  the 
USVI  plot  (42.5%)  was  higher  than  the  coral  cover  at  the  Dry  Tortugas  plot  (23.1%).  Plating  or 
encrusting  forms  of  colonies  of  the  genus  Montastraea  dominated  both  communities.  The  mean 
relative  cover  of  this  genus  was  85%  at  the  USVI  and  54%  at  the  Dry  Tortugas.  Other  coral 
genera  found  at  lower  abundances  in  both  plots  included  Agaricia,  Colpophyllia,  Helioseris, 
Mycetophyllia,  and  Porites.  The  dominant  macroalgae  was  Lobophora  variegata ,  seen  growing 
on  dead  coral  skeletons  at  both  locations.  The  cover  of  both  soft  corals  and  sponges  was  low  at 
both  sites  (<  2  %).  Measurements  of  coral  cover  derived  from  the  mosaics  are  within  one 
standard  deviation  of  diver  line-intercept  transects  at  and  within  500  m  of  the  Dry  Tortugas  site 
(25.4  %,  ±8.1  S.D.)  and  diver  video  transects  at  and  within  1  km  of  the  USVI  site  (41.0  %,  ±1.9 
S.D.). 
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Figure  A52.  Site  one  from  Sherwood  Forest  Reef,  Dry  Tortugas,  Florida  (23m  water  depth). 
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Figure  A53.  Site  two,  a  coral  hillock  from  St.  Thomas  USVI  (38.5m  water  depth). 


In  this  study,  the  software  previously  used  to  create  landscape  mosaics  for  shallow  coral  reefs 
was  applied  to  mesophotic  coral  reef  ecosystems.  Landscape  mosaics  are  well  suited  to  this 
environment,  in  part  due  to  the  plating  and  encrusting  growth  forms  of  coral  colonies  at 
mesophotic  depths  that  lend  themselves  to  down-looking  image-based  monitoring. 


One  benefit  of  landscape  mosaics  is  that  they  capture  a  permanent  record  from  which  percent 
cover,  size,  diversity  indices,  and  the  status  of  hundreds  of  individual  colonies  may  be  extracted 
as  needed.  A  second  benefit  of  landscape  mosaics,  like  other  image-based  survey  methods,  is 
that  shifting  analysis  to  the  lab  saves  dive  time,  which  is  critical  at  the  depths  of  mesophotic  reef 
ecosystems.  Dive  time  and  diver  training  were  not  obstacles  to  video  acquisition  for  landscape 
mosaics,  which  in  each  case  took  25  minutes.  Different  divers  acquired  the  footage  for  each 
mosaic.  Although  both  were  experienced  science  divers,  the  diver  for  the  Dry  Tortugas  survey 
had  never  acquired  footage  for  video  mosaics  before,  and  the  diver  for  the  USVI  survey  had 
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made  just  one  practice  survey.  The  ease  of  data  acquisition,  in  combination  with  the  ability  to  use 
off-the-shelf  hardware  that  most  coral  reef  ecologists  probably  have  already  (i.e.  a  video  camera 
and  possibly  lights),  suggests  that  landscape  mosaics  could  be  applied  widely  for  inexpensive 
surveys  of  mesophotic  coral  ecosystems. 

A  portion  of  the  above  work  was  presented  in  the  following  paper: 

Gleason,  A.  C.  R.,  N.  Gracias,  D.  Lirman,  B.  E.  Gintert,  T.  B.  Smith,  M.  C.  Dick,  R.  P.  Reid 
(2009).  Landscape  video  mosaic  from  a  mesophotic  coral  reef,  Coral  Reefs,  doi: 
1 0. 1 007/s00338-009-0544-2. 

A2.4.4  Using  Landscape  Mosaics  to  Assess  the  Impacts  of  Hurricane  Damage  on  Acropora 
palmata  populations 

To  document  the  impacts  of  storm  damage  on  populations  of  Acropora  palmata  we  used  video¬ 
mosaic  technology  to  document  a  population  of  the  branching  coral  Acropora  palmata  at 
Molasses  Reef  (25°  0.609,  N  80°  22.397  W,  depth  =  3. 5-4. 5  m)  both  before  and  after  the  2005 
hurricane  season. 

Mosaics  of  the  study  plot  (approximately  10  m  x  10  m)  were  constructed  from  underwater  video 
collected  before  and  after  the  2005  hurricane  season  (Figure  A54). 


Figure  A54.  Video  mosaics  from  a  study  plot  at  Molasses  Reef  in  the  Florida  Reef  Tract  (depth=  3.5  -  4.5  m). 
A)  Mosaic  from  May  2005,  prior  to  the  start  of  the  2005  hurricane  season.  B)  Mosaic  from  February  2006 
following  the  passage  of  Hurricanes  Dennis  (July  9-10),  Katrina  (August  25-26),  Rita  (September  19-20),  and 
Wilma  (October  24-25).  The  yellow  line  A-B  shows  where  the  reef  framework  was  dislodged  during 
hurricane  Rita  causing  sections  of  the  reef  marked  C  and  D  to  collapse.  The  section  labeled  C  also  appears  in 
the  figure  bellow. 

Video  data  were  collected  at  Molasses  Reef  in  May  2005  and  in  February  2006  after  the  passage 
of  Hurricanes  Dennis  (dates  of  influence  over  the  Florida  Keys  =  July  9-10,  peak  wind  gusts  at 
Molasses  Reef  (C-MAN  station)  =  90  kph),  Katrina  (August  25-26,  116  kph),  Rita  (September 
19-20,  100  kph)  and  Wilma  (October  24-25,  147kph). 
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The  video-mosaics  produced  in  this  study  have  high  spatial  accuracy  and  thereby  provide  the 
capability  to  measure  distances  and  sizes  directly  from  the  images  once  a  scale  has  been 
established  (Lirman  et  al.  2006).  The  scale  in  these  mosaics  is  provided  by  PVC  segments  and 
ceramic  tiles  scattered  throughout  the  images.  The  size  of  the  A.  palmata  colonies  found  within 
each  mosaic  was  measured  as:  (1)  the  maximum  colony  diameter;  and  (2)  the  projected  surface 
area  of  live  tissue  using  the  image-analysis  software  Image-J. 

The  direct  physical  damage  caused  by  hurricanes  and  tropical  storms  can  vary  significantly 
across  multiple  spatial  scales,  ranging  from  minimal  to  severe  (Harme line -Vivien  1997). 
Whereas  changes  in  coral  cover,  abundance,  and  condition  can  be  easily  discerned  from 
traditional  before-and  after  surveys,  changes  to  the  structure  of  reefs  are  harder  to  quantify.  The 
video  mosaics  created  in  this  study  provide  a  unique  view  of  the  reef  benthos  that  facilitates  the 
documentation  of  colony-level  impacts  as  well  as  large-scale  structural  changes  to  the  reef 
framework. 

If  only  colony-based  information  such  as  coral  cover,  abundance,  and  size- structure  had  been 
collected  prior  to  the  onset  of  the  2005  hurricane  season,  the  damage  report  for  the  A.  palmata 
population  at  Molasses  Reef  would  have  revealed  limited  damage.  A  total  of  19  A.  palmata 
colonies  were  identified  from  the  video  mosaics  from  May  2005,  prior  to  the  onset  of  the  2005 
hurricane  season,  and  17  of  these  colonies  remained  (in  the  same  location)  in  the  study  plot  in 
February  2006  (Figure  A54).  The  two  colonies  that  were  removed  from  the  plot  were  located  on 
the  sections  of  the  reef  framework  that  was  dislodged  during  Hurricane  Rita  (Figure  A55.a). 
These  two  colonies  remained  attached  to  the  dislodged  reef  section  but  ended  up  in  contact  with 
bottom  sediments  and  died  shortly  after  the  storm  (Figure  A55.b).  The  tissue  on  these  large 
colonies  (110  and  115cm  in  maximum  diameter)  represented  14%  of  the  total  live  Acropora 
tissue  on  the  plot  prior  to  the  storms.  For  those  colonies  that  remained,  the  net  tissue  losses 
between  surveys  were  only  10%.  Fifty-two  percent  of  colonies  lost  live  tissue;  the  maximum 
tissue  loss  for  an  individual  colony  was  46%,  and  the  maximum  tissue  gain  was  47%  of  the 
original  tissue  area.  The  mean  diameter  of  colonies  decreased  slightly  from  88.4cm  (SD  ±70.1) 
to  79.6  (±63.3)  cm.  Tissue  losses  were  mainly  attributed  to  the  removal  of  branches  (Figure 
A55.C,  d). 
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Figure  A55.  a)  Photograph  of  the  reef  section  dislodged  during  Hurricane  Rita,  b)  Photograph  of  two  A. 
palmata  colonies  attached  to  the  dislodged  reef  section.  These  colonies  ended  up  facing  the  sediments  and 
died  shortly  after  the  storm.  Sections  of  video  mosaics  from  May  2005  (c)  and  Feb  2006  (d)  showing  an  A. 
palmata  colony  (labeled  E  in  the  figure  above)  that  experienced  fragmentation  and  tissue  mortality  due  to  the 
2005  hurricanes. 


Previous  studies  have  documented  an  increase  in  the  abundance  of  A.  palmata  colonies  through 
fragment  formation  and  reattachment  after  storms  (Fong  and  Lirman  1995).  This  was  not 
observed  within  the  2005-2006  study  plot  at  Molasses  Reef.  Fragment  reattachment  requires 
time  (Lirman  2000)  and  the  succession  of  storms  during  the  summer  of  2005  may  have  impeded 
this  process. 

Although  there  was  limited  impact  from  the  2005  storms  on  individual  coral  colonies,  large 
changes  to  the  reef  framework  were  documented  at  this  site.  Within  the  study  plot,  a  large 
section  of  the  reef  (surface  area  =  12.7  m2,  diameter  =  6m)  was  dislodged  and  deposited  on  the 
sand  at  the  bottom  of  the  reef  spur  (Figure  A54,  A55a).  The  shift  in  orientation  of  these  sections 
resulted  in  the  smothering  and  burial  of  coral  colonies  and  the  exposure  of  reef  framework, 
which  may  be  further  weakened  by  the  future  activities  of  bioeroders  (Glynn  1988).  The  precise 
documentation  of  such  large-scale  modifications  to  the  structure  of  the  reef  would  not  have  been 
possible  without  the  availability  of  the  landscape  view  provided  by  the  video-mosaics. 

The  methods  used  to  assess  damage  and  recovery  patterns  of  reef  communities  commonly  entail 
the  construction  of  underwater  maps  of  the  benthos  based  on  diver-collected  distance 
measurements  and  drawings,  and  the  deployment  of  survey  markers  and  permanent  tags  for  coral 
colonies  within  plots.  Assessing  the  impacts  of  severe  physical  disturbance  on  coral  reefs  can  be 
especially  challenging  when  large-scale  modifications  to  the  reef  structure  takes  place  and  both 
coral  colonies  and  survey  markers  are  removed  (Hudson  and  Diaz  1988;  Jaap  2000).  The  data 
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presented  in  this  study  show  that  landscape  video-mosaics  provide  the  tools  needed  to  accurately 
assess  reef  damage  and  recovery  patterns  at  multiple  scales  and  represent  a  significant 
improvement  over  existing  survey  techniques,  which  require  extensive  bottom  time  and  the  use 
of  trained  research  divers. 

The  above  research  was  included  in  the  following  publication: 

Gleason,  A.C.,  Lirman,  D.,  Williams,  D.,  Gracias,  N.R.,  Gintert,  B.E.,  Madjidi,  H.,  Reid,  R.P., 
Boynton,  G.C.,  Negahdaripour,  S.,  Miller,  M.  and  Kramer,  P.  (2007)  Documenting  hurricane 
impacts  on  coral  reefs  using  two-dimensional  video-mosaic  technology.  Marine  Ecology  28:1-5. 


A2.4.5  Use  of  2D  Video  Mosaics  for  Assessing  the  Impacts  of  Mass-Bleaching  Events  on 
Coral  Communities 

Our  efforts  in  2006  showed  the  benefits  of  mosaics  in  assessing  the  impact  of  mass  bleaching 
events  on  coral  communities.  In  2005,  Caribbean  reefs  experienced  a  major  bleaching  event  that 
resulted  in  significant  coral  mortality  throughout  the  region.  The  magnitude  of  the  2005 
bleaching  event  has  prompted  a  federal  response  by  the  US  Coral  Reef  Task  Force  to  initiate 
large  scale  assessments  of  short  and  long  term  impacts  of  this  event  on  Caribbean  coral 
communities. 

During  December  2005,  we  surveyed  a  shallow  patch  reef  offshore  of  the  AUTEC  naval  base  on 
Andros  Island,  Bahamas.  Although  our  survey  was  not  conducted  at  the  peak  of  the  bleaching 
event  (August-October),  bleached  colonies  were  still  observed  at  this  site.  2D  video  mosaics 
were  made  from  video  footage  acquired  in  December  of  2005  and  again  in  October  2006. 

By  rectifying  the  resulting  mosaics,  pale  or  bleached  colonies  identified  in  the  2005  mosaic  were 
relocated  in  the  October  2006  mosaic  to  document  patterns  of  coral  mortality  and/or  recovery 
(Figure  A56.a).  The  analysis  of  these  mosaics  indicates  that  bleaching  status  in  2005  was  not  a 
good  predictor  of  mortality  as  examples  of  both  partial  mortality  and  colony  recovery  occurred 
within  the  site  (Figure  A56.b  &  c).  Several  of  the  coral  colonies  that  were  completely  bleached 
in  December  2005,  had  recovered  their  pigmentation  by  October  2006  (Figure  A56.c),  while 
other  colonies  that  showed  only  slight  paling  in  December  2005  showed  significant  partial 
mortality  8  months  later  (Figure  A56.c). 

The  inability  to  use  bleaching  severity  as  a  predictor  of  coral  recovery  potential  highlights  the 
appeal  of  using  landscape  video  mosaics  that  capture  the  health  of  all  colonies  >5  cm  within  a 
given  area  to  follow  changes  over  time.  This  feature  allows  users  to  follow  individual  colonies 
through  time  with  limited  field  effort.  Although  standard  tagging  based  approaches  that 
randomly  tag  large  numbers  of  colonies  will  yield  similar  information  on  the  health  of  bleached 
and  non-bleached  colonies  over  time,  these  approaches  are  time  consuming  underwater  and  are 
often  plagued  by  tag  loss  that  can  cause  coral  health  information  to  be  lost. 
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December  2005  October  2006 


Figure  A56.  (a)  Landscape  video  mosaics  of  site  Sl-10  located  offshore  of  Andros  Island,  Bahamas  in 
December  2005  (left  column)  and  October  2006  (right  column).  Affected  colonies  were  identified  using  the  2D 
video  mosaics  shown  in  (a).  Red  arrows  indicate  colonies  that  underwent  partial  mortality  between  December 
2005  and  October  2006.  Blue  arrows  indicate  colony  recovery  during  the  same  time  period. 

The  application  of  our  2D  mosaic  products  to  the  assessment  of  mass  bleaching  events 
indicates  that  this  technology  provides  a  large  scale  permanent  record  of  colony  location,  and 
health  that  can  be  followed  over  time  to  assess  coral  colony  mortality  and  recovery  without 
extensive  field  efforts. 
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A2.5  Users/Partners 

Throughout  the  term  of  this  project  our  group  has  developed  several  collaborations  with  both 
government  and  scientific  users  of  landscape  video  mosaics.  The  results  of  these  collaborations 
are  outlined  below: 

A2.5.1  SERDP  Coral  Reef  Monitoring  and  Assessment  Workshop 

A  SERDP-sponsored  workshop  was  held  at  the  Rosenstiel  School  of  Marine  and  Atmospheric 
Science,  University  of  Miami  Nov  18-19,  2008.  The  primary  focus  for  the  workshop  was  to:  (1) 
understand  the  DoD  client  perspective  on  assessment  and  monitoring  needs;  (2)  understand  other 
potential  user  perspectives  (i.e.,  in  addition  to  DoD)  regarding  their  coral  reef  monitoring  and 
assessment  needs  and  how  the  two  SERDP-developed  technologies  (landscape  mosaics  and 
FIRe)  may  help  address  those  needs;  and  (3)  identify  how  the  two  approaches/technologies  are 
complementary  to  each  other  and  how  they  might  be  integrated  to  meet  end-user  needs. 

Presentations  by  DoD  personnel,  representatives  from  key  governmental  and  non-governmental 
organizations/offices  actively  involved  in  coral  reef  management  and  research,  and  the  research 
teams  from  the  University  of  Miami  and  Rutgers  were  interspersed  with  active  discussion. 
Results  from  the  workshop  are  summarized  in  a  comprehensive  report  (SPAWARSYSCEN 
PAC.  2008).  Key  findings  include  the  following: 


1)  Federal  policy  mandates  that  DoD  characterize,  assess,  and  monitor  underwater 
benthic  communities  at  Air  Force,  Army,  and  Navy  bases  in  order  to  document 
compliance  with  national  policy  and  to  ensure  that  DoD  operations  do  not  lead  to  natural 
resource  degradation,  particularly  with  respect  to  coral  reefs.  DoD  is  looking  for 
technologies  and  methodologies  that  will  enable  the  collection  of  the  data  with  less  dive 
time,  reproduce  data  collection  transects  reliably  year  after  year,  and  retain  flexibility  to 
be  modified  based  on  expert  evaluation  of  site  conditions  at  the  time  of  the  survey.  DoD 
is  also  interested  in  exploring  how  emerging  technologies  may  foster  new  opportunities 
to  develop  productive  partnerships  between  the  NAVY  and  other  organizations. 


2)  Workshop  participants  were  in  agreement  that  current  monitoring  and  assessment 
strategies  conducted  by  the  agencies  are,  in  general,  adequate  to  meet  present  mandates, 
but  there  was  broad  interest  from  all  agencies  in  developing  methodologies  that  reduce 
dive  time  and  cost  efficiency.  Specific  challenges  and  needs  to  expand  present 
methodologies  include  capabilities  for  detailed  mapping  with  improved  accuracy 
compared  to  currently  used  strip  (ID)  mosaics,  monitoring  deep  reefs,  studies  of  infection 
patterns  of  coral  disease,  and  non  destructive  methods  for  determining  coral  physiology. 


3)  There  was  consensus  regarding  the  usefulness  of  landscapes  mosaics  and  FIRe 
technologies  for  advancing  coral  reef  monitoring  and  assessment  practices.  Individually 
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or  combined,  these  techniques  offer  potential  for  more  efficient  methods  of  monitoring 
coral  cover,  colony  size,  mortality,  bleaching  and  disease  prevalence,  coral  population 
structure,  extent  of  injury  and  recovery  patterns.  There  was  also  agreement  that  the 
transition  of  both  technologies  to  the  end-user  community  would  be  valuable. 


The  potential  for  the  two  SERDP  technologies  to  augment  and  enhance  the  specific  reef 
monitoring  and  assessment  activities  of  the  participating  agencies  is  summarized  in  Table  A 12. 
Column  1  of  Table  A12  lists  the  governmental  and  non-governmental  agencies  represented  by 
workshop  presenters  and  other  participants.  Columns  2,  3  and  4  are  color  coded  to  indicate 
potential  contributions  of  mosaics  (green),  FIRe  (yellow)  or  both  technologies  (purple)  to 
augment  or  enhance  monitoring  of  present  metrics  (Column  2),  enable  new  desired  capabilities 
(Column  3),  or  provide  new  opportunities  for  partnerships  (Column  4).  Text  in  Column  2 
identifies  indices  of  reef  health  presently  monitored  by  each  agency  that  could  benefit  from  the 
use  of  mosaics  and/or  FIRe.  Text  in  Column  3  identifies  desired  enhanced  monitoring 
capabilities  that  could  be  accomplished  using  mosaics  and/or  FIRe.  Column  4  summarizes 
potential  collaborations  using  mosaics  and/or  FIRe. 

Table  A12.  Technology  overlay  and  potential  collaborations  resulting  from  SERDP  coral  reef  monitoring 
and  assessment  workshop. 

Potential  roles  for:  Mosaicsf  |  FlRe|  ~|  Mosaics  &  FIRe  \~  [ 


Agency 

Current  Metrics /Indicators 

iDesired  Capabilities 

Collaborations 

Presenters 

US  Navy 

coral  cover,  colony  size,  disease  & 
bleaching,  mortality 

increased  sampling  efficiency;  multi-tier 
approach:  safe,  efficient  cost  effective: 
digital  technology: 

MMS 

coral  cover,  diversify,  coral  growth 

reef  condition  S  coral  growth,  Flower  Gardens 

NPS 

coral  cover,  disease  &  bleaching 

detailed  mapping;  disease  causes  and 
infection  research 

coral  monitoring  program,  Biscayne  National 
ParR 

FWS 

coral  cover,  diversity,  disease  & 
bleaching 

improved  monitoring  techniques  and 
technologies 

monitoring  at  remote  refuges,  e.g.  Palmyra 
Island 

ERA 

coral  cover  (2D  &  3D),  colony  size, 
disease  &  bleaching,  mortality, 
population  structure 

expand  surveys  to  include  additional  benthic 
organisms  and  determine  links  to  stressors 

investigate  sampling  efficiency  in  different 
environments  (mosaics):  assess  coral  viability, 
identify  stressors,  and  collect  database  of  FI  Re 
signatures 

NOAA  Fisheries 

coral  cover,  colony  size,  disease  & 
bleaching,  mortality 

reduce  dive  time  and  expand  surveys  to 
deep  reefs  (e.g.,  Oculina  banks) 

survey  deep  coral  communities  (i.e.,  Qcufina 
banks)  and  thickets  of  thretened  corals  (i.e., 
Acropora) 

NOAA  Coastal 
Monitoring  and 
Assessement 

biomaiRers,  coral  diseases 

rapid,  effective,  non-destiuctive  methods  to 
evaluate  coral  physiological  condition 

use  in-situ  FIRe  in  combination  with  chemical, 
microbiological  a  biomarker  sampling  to  assess 
coral  response  to  mixed  stressors. 

NOAA 

Sanctuaries 

injury  &  %  cover  at  damaged  and 
reference  sites,  recovery  patterns 

increase  survey  spped;  reduce  need  for 
trained  divers;  improvements  relative  to  strip 
mosaics 

groundings  assessment,  FKNMS 

NOAA 

Restoration 

coral  colony  size,  partial  mortality, 
disease  &  bleaching  %  cover,  urchin 
abundance 

survey  deep  reefs  (>  30  m) 

survey  coral  reef  and  seagrass  sites  to  compare 
survey  methods,  products,  a  cost  effectiveness 

TNG 

coral  cover,  disease  &  bleaching, 
colony  size,  mortality 

identify  causes  of  changes  in  coral  condition 
and  demographics  occurring  between 
annual  sampling  events 

continue  ongoing  collaboration  with  the  Florida 
Reef  Resilence  Program 

Other  participants 

US  NavyAUTEC 

continue  ongoing  collaboration  mosaicing 
permanent  monitoring  sites 

NCRI 

address  pixel  mixing  problem  in  airborne 
remote  sensing  images 

Florida  Sea  Grant 

surveys  at  CREWS/ICON  stations 

Following  the  SERDP  coral  reef  monitoring  and  assessment  workshop  several  collaborations 
with  reef  monitoring  user  groups  were  created  and/or  extended.  Overviews  of  our  collaborative 
efforts  are  described  in  the  following  sections. 
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A2.5.2  The  National  Park  Service:  Surveys  of  the  threatened  coral  Acropora  palmata 

In  conjunction  with  the  National  Park  Service  in  St.  Croix,  USVI,  mosaic  surveys 
were  completed  at  four  sites  within  the  Buck  Island  Reef  National  Monument. 
Sites  were  chosen  to  document  the  current  state  of  the  threatened  coral  species 
/  "v  *  Acropora  palmata  at  Buck  Island  reefs  and  provide  baseline  data  from  which 
impacts  of  future  hurricanes,  bleaching,  and  disease  events  could  be  assessed. 
These  sites  at  Buck  Island  are  known  to  be  some  of  the  best  remaining  Acropora 
palmata  reefs  in  the  Caribbean.  The  four  reefs  surveyed  varied  in  terms  of 
exposure  and  depth  providing  a  comprehensive  survey  of  the  types  of  Acropora  palmata  reefs 
present  at  Buck  Island. 


Figure  A57.  2D  video  mosaics  of  Acropora  palmata  reefs  at  Buck  Island  National  Monument,  St.  Croix, 
USVI.  The  reef  in  (A)  is  a  deep  patch  reef  consisting  of  very  large  individual  colonies  of  Acropora  palmata. 
B)  Is  a  shallow  inshore  patch  reef  site. 


A2.5.3  University  of  Puerto  Rico 


In  December  2007  we  were  invited  to  participate  in  a  large  multi-organizational 
hyperspectral  mission  headed  by  the  University  of  Puerto  Rico  at  Mayaguez 
(UPRM).  As  part  of  this  mission,  different  resolution  airborne  hyperspectral 
imagery  was  acquired  over  designated  reef  science  areas.  During  the  mission 
2D  video  mosaics  were  acquired  in  four  different  reef  environments  to  provide 
spatially  explicit  groundtruth  information  for  the  hyperspectral  images. 
Additional  spectral  data  and  precision  gps  information  was  taken  during  mosaic  acquisition.  The 
image  based  groundtruthing  will  allow  for  an  in-laboratory  assessment  of  the  accuracy  of 
hyperspectral  image  classification  at  various  resolutions.  This  is  the  first  attempt  at  using  2D 
video  mosaicing  technology  to  groundtruth  hyperspectral  imagery. 
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A2.5.4  The  US  Navy’s  Coral  Reef  Monitoring  Program  at  AUTEC 

In  association  with  the  US  Navy’s  coral  reef  monitoring 
program  at  the  Atlantic  Undersea  Test  and  Evaluation  Center 
(AUTEC)  we  have  begun  using  2D  video  mosaics  to  monitor  a 
number  of  their  1  Ox  10m  permanent  reef  monitoring  sites. 
Installed  in  the  late  1960’s  these  sites  provide  a  long  history  of 
reef  health  information.  2D  video  mosaics  have  been  acquired 
repeatedly  at  a  number  of  these  sites  between  2005  and  2009. 

As  a  result  of  this  monitoring  effort  we  have  been  able  to  monitor  Andros  Island  coral 
populations  following  the  2005  Caribbean-wide  mass  bleaching  event,  the  2009  region-wide 
bleaching  and  disease  event,  and  assess  the  spatio-temporal  dynamics  of  Andros  Island  coral 
communities  over  the  last  40  years  (See  sections  A5.4.2  and  A5.4.5). 


Blow-up  of  10MP  Still 
Image 


Integrated  10MP  Still 
Image 


Figure  A58.  Example  of  coral  reef  monitoring  at  Andros  Island,  Bahamas.  In  the  images  above  all  coral 
colonies  within  a  lOxlOrn  permanent  reef  plot  were  surveyed  and  monitored  during  the  2009  bleaching  and 
disease  event  using  second-generation  video  mosaicing  technology.  This  high  resolution  imagery  allowed  for 
the  identification  of  diseased  coral  colonies  (such  as  the  one  shown  above)  without  the  use  of  tagging. 
Colonies  at  this  site  have  been  continuously  monitored  for  changes  since  the  inception  of  the  collaboration  in 
2005. 
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A2.5.5  The  Nature  Conservancy 


TheNature 

Conservancy 

Protecting  nature.  Preserving  life." 


In  collaboration  with  the  Florida  Reef  Resilience  Program  we 
proposed  to  construct  video  mosaics  of  reef  plots  (5  x  5  m) 
within  four  reef  habitats  (i.e.,  inshore,  mid-channel,  fore-reef, 
and  offshore  patch  reef  habitats)  to:  (1)  test  the  potential 
application  of  this  methodology  in  documenting  reef 
resilience  and  resistance  to  bleaching  impacts;  and  (2) 
establish  eight  permanent  plots  (replicated  by  habitat  type)  to  monitor  future  temporal,  colony- 
based  patterns  of  bleaching,  mortality,  and  recovery. 


As  part  of  the  Florida  Reef  Resilience  Program’s  yearly  bleaching  surveys,  we  identified  eight 
sites  (n  =  2  per  habitat  type)  where  permanent  plots  (5  m  x  5  m)  were  established  by  hammering 
and  cementing  sections  of  rebar  into  the  reef  substrate.  The  center  of  each  plot  was  marked  with 
a  buoy  and  a  GPS  location  was  established  from  the  surface  (Figure  A59).  At  each  site,  digital 
video  and  still  images  were  collected  at  1.5  -  2  m  from  the  bottom. 


Figure  A59.  Location  of  the  permanent  sites  established  as  part  of  the  bleaching-mosaic  assessment  of  the 
Biscayne  Subregion  of  the  Florida  Reef  Tract  in  the  summer  of  2008.Reef  habitats  appear  in  yellow.  Red  = 
Inshore  sites,  green  =  mid-channel  sites,  blue  =  fore-reef  sites,  black  =  offshore  patch  reef  sites.  Triangles 
mark  the  locations  of  the  permanent  sites  for  which  video  mosaics  were  constructed  (Figs.  A60-A63). 


For  four  of  the  eight  permanent  plots,  video  mosaics  were  created  using  the  algorithm  described 
in  detail  by  Lirman  et  al.  (2007).  The  four  mosaics  constructed  for  this  study  have  a  ground 
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resolution  of  1-2  mm.pixeF1  (Figs.  A60-A63).  These  video  mosaics  cover  areas  of  roughly  100- 
125  m2  and  allow  for  the  tracking  of  hundreds  of  coral  colonies  without  the  need  to  tag 
individual  colonies.  The  spatial  accuracy  of  the  mosaics  also  provide  a  way  to  measure  coral 
sizes  and  patterns  of  tissue  mortality  that  would  be  important  indicators  of  individual  and 
population  responses  to  stress.  Future  surveys  require  only  that  divers  relocate  the  permanent 
markers  and  video  surveys  can  be  easily  completed  in  <  30  min  of  bottom  time. 


Figure  A60.  Landscape  video  mosaic  of  a  permanent  plot  established  within  an  Offshore  Patch  Reef  habitat 
as  part  of  the  Florida  Reef  Resilience  Program  in  the  summer  of  2008.  Site  number  =  1188,  Site  location  =  25° 
24.088’  N,  80°  9.208’  W,  depth  =  12  m. 
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Figure  A61.  Landscape  video  mosaic  of  a  permanent  plot  established  within  a  Mid-Channel  Patch  Reef 
habitat  as  part  of  the  Florida  Reef  Resilience  Program  in  the  summer  of  2008.  Site  number  =  2197,  Site 
location  =  25°  30.525’  N,  80°  7.232’  W,  depth  =  4  m. 
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Figure  A62.  Landscape  video  mosaic  of  a  permanent  plot  established  within  an  Inshore  Patch  Reef  habitat  as 
part  of  the  Florida  Reef  Resilience  Program  in  the  summer  of  2008.  Site  number  =  3161,  Site  location  =  25° 
27.  819’  N,  80°  10.085’  W,  depth  =  3  m. 
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Figure  A63.  Landscape  video  mosaic  of  a  permanent  plot  established  within  a  Fore-Reef  habitat  as  part  of  the 
Florida  Reef  Resilience  Program  in  the  summer  of  2008.  Site  number  =  3202,  Site  location  =  25°  40.508’  N, 
80°  5.908’  W,  depth  =  11  m. 


While  the  principal  objective  of  this  project  was  to  exploit  the  benefits  of  mosaic  images  to  track 
spatial  patterns  of  coral  bleaching  response  and  recovery,  no  significant  coral  bleaching  was 
observed  in  2008  (only  a  few  colonies  showed  signs  of  bleaching  within  the  plots  established; 
Figs.  A60-A63).  Nevertheless,  the  eight  permanent  sites  established,  the  video  and  stills 
collected,  and  the  four  mosaics  created  provide  the  tools  and  resources  needed  to,  in  the  event  of 
a  future  bleaching  event,  document  bleaching  impacts  rapidly  and  accurately  and  establish  a 
baseline  against  which  patterns  of  mortality  and  recovery  (i.e.,  resilience)  can  be  accurately 
assessed  over  time. 

The  four  mosaic  sites  surveyed  in  2008  were  re-visited  in  the  summer  of  2009.  Bleaching 
prevalence  in  2009  (as  was  the  case  in  2008)  was  limited.  Nevertheless,  a  small  number  of 
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colonies  within  the  survey  plots  exhibited  signs  of  bleaching.  The  coral  colonies  showing  signs 
of  paling  or  bleaching  were  identified  and  marked  with  numbered  tiles  that  appear  in  the 
mosaics,  tagged  with  numbered  aluminum  tags,  and  measured.  These  tiles  were  deployed  within 
the  area  appearing  in  the  mosaic  and  thus  provide  a  way  of  finding  and  identifying  these  colonies 
for  future  assessments  (Figure  A64).  Subsequent  mosaic  surveys  will  enable  us  to  assess  the 
patterns  of  growth,  survivorship,  and  mortality  of  these  bleached  colonies,  thus  providing  a  good 
tool  to  evaluate  bleaching  impacts  and  reef  resilience  for  the  different  reef  habitats  of  the  Florida 
Reef  Tract. 


Figure  A64.  Landscape  mosaic  of  a  Fore-Reef  habitat  showing  the  location  of  bleached  colonies  of  Siderastrea 
siderea.  Bleached  colonies  within  mosaics  were  identified  with  a  numbered  or  colored  tile. 


A2.5. 6  Rutgers  University 


As  part  of  the  project  extension,  we  were  asked  to  collaborate 
with  Rutgers  University  to  evaluate  the  potential  for  integrating 
landscape  mosaic  technology  with  another  methodology 
developed  under  SERDP:  Fluorescence  Induction  and 
Relaxation  technique,  or  FIRe,  developed  by  Drs.  Maxim 
Gorbunov  and  Paul  Falkowski  at  Rutgers. 

On  June  16-18,  2008,  researchers  from  the  University  of  Miami  and  Rutgers  University 
conducted  a  joint  sampling  effort  in  the  northern  section  of  the  Florida  Reef  Tract  to  evaluate  the 
potential  for  integrating  these  two  distinct  coral  monitoring  methodologies  into  an  integrated 
sampling  strategy.  As  an  initial  evaluation  of  this  potential  integrated  approach,  two  coral  reefs 
sites  were  selected  and  surveyed  jointly.  At  each  reef,  high-resolution  video  and  still  images 
were  collected  from  a  reef  area  roughly  10  x  10  m.  The  data  collected  will  be  used  to  construct  a 
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spatially  accurate,  high-resolution  video  mosaic  using  the  methods  described  by  Lirman  et  al., 
2007.  In  addition,  20-30  benthic  organisms  dominant  in  the  area  (including  hard  corals,  soft 
corals,  sponges,  macroalgae)  were  identified  within  the  imaged  plot  and  FIRe  fluorescence 
measurements  were  taken  for  each  organism  using  the  diver-operated  FIRe  instrument  as 
described  in  (Gorbunov  et  al,  2001;  Gorbunov  and  Falkowski,  2005). 

The  first  reef  surveyed,  Brooke’s  Reef,  is  located  at  25  40.508°N,  80  5.908°W.  This  reef  is  found 
at  10  m  of  depth  and  it  has  been  surveyed  annually  since  2004  by  University  of  Miami 
researchers  using  the  video  mosaic  technology.  The  plot  surveyed  within  Brooke’s  Reef  is 
demarcated  by  four  pipe  sections  permanently  attached  to  the  bottom.  These  markers  are  easily 
identified  within  the  video  mosaic,  which  allows  for  multiple  mosaics  to  be  referenced  against 
each  other  for  change-detection  analyses.  Prior  to  the  collection  of  the  video  imagery  for 
constructing  the  video  mosaics,  30  benthic  organisms  were  identified  within  the  plot  by 
deploying  numbered  tiles.  These  tiles  are  commonly  used  to  identify  organisms  or  features 
within  the  completed  video  mosaics.  In  this  case,  the  tiles  will  be  used  to  link  the  organisms 
within  the  video  mosaic  with  the  physiological  measurements  obtained  for  each  one.  After  the 
completion  of  the  image  acquisition  (—30  min),  researchers  from  Rutgers  University  collected 
fluorescence  measurements  for  each  of  the  organisms  identified  by  the  numbered  tiles.  The  time 
required  to  collect  the  fluorescence  information  from  30  objects  (with  3-4  replicates  on  each 
target)  was  approximately  40-60  min. 

The  same  procedure  was  repeated  at  a  second,  shallower  reef  (5  m)  (25  35.255°N  80  6.778°W). 
In  this  case,  a  permanent  plot,  roughly  5  m  x  5  m,  was  established  by  cementing  rebar  stakes 
onto  the  reef  bottom.  Within  this  plot,  20  benthic  organisms  were  identified  with  numbered  tiles 
as  described,  and  video  images  were  collected.  As  described,  fluorescence  measurements  of  all 
of  the  tagged  benthic  organisms  were  collected  after  the  collection  of  the  video.  The  two  research 
groups  worked  well  as  a  team  and  permanent  sites  were  established  and  surveyed  within  two 
hours  of  bottom  time. 

The  results  of  the  field  experiment  resulted  in  a  combined  mosaic-FIRe  product  as  shown  in 
Figure  A65. 
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Figure  A65.  Combined  mosaic/FIRe  product.  Locations  of  individual  organisms  examined  with  the 
underwater  FIRe  instrument  are  demarcated  by  the  triangles  (above).  The  color  of  the  triangles  indicates  the 
relative  photosynthetic  stress  on  the  organism  as  determined  by  Dr.  Gorbunov  (green=  healthy;  yellow= 
potential  stress;  red=  photosynthetic  stress). 

Individual  FIRe  measurements  of  corals,  sponges,  and  gorgonians  were  spatially  mapped  by 
matching  numbers  from  tiles  placed  near  the  test  organism  at  the  time  of  mosaic  acquisition  to 
those  indicated  by  diver  notes.  Measurements  were  evaluated  by  Dr.  Gorbunov  and  given  a 
relative  health  index  based  on  the  5  measurements  obtained  from  individual  FIRe  measurements 
(F0,  Fm,  Fv/Fm,  sigma,  and  Taul).  Health  indices  were  separated  into  three  groups;  green,  yellow 
and  red  based  on  their  FIRe  values.  Green  indicated  limited  stress  on  the  target  organism,  yellow 
indicated  possible  stress,  and  red  indicated  significant  stress  on  the  target  organism. 

Analysis  of  the  combined  products  was  difficult  due  to  the  limited  number  of  sample  points  at 
each  site.  Despite  this  limitation  a  vision  of  a  future  integrated  product  was  developed  as  part  of 
the  collaboration  (shown  below  in  Figure  A66). 
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Figure  A66.  Future  of  integrated  mosaic/FIRe  survey  technology.  1)  Vision  of  integrated  platform  with 
second-generation  mosaic  system  combined  with  new  distance  FIRe  instrument.  Both  will  be  acquired 
simultaneously  at  a  distance  of  2m  from  the  bottom.  2).  Results  of  integrated  instrumentation.  High 
resolution  mosaic  products  will  be  combined  with  100’s  to  thousands  of  FIRe  measurements  taken  during  a 
single  survey.  3).  Using  information  from  the  combined  product  we  can  produce  dense  benthic  health  maps 
to  locate  hot  spots  of  stress  in  coral  environments. 


The  possible  combination  of  the  two  survey  techniques  provides  a  significant  enhancement  to 
current  reef  monitoring  technology  by  providing  three  levels  of  reef  health  information: 
landscape  level  (mosaic  products),  colony  level  (integrated  stills)  and  physiological  (FIRe)  from 
a  single  survey  platform.  Our  vision  of  the  future  technology  allows  for  a  diver  to  swim  a  single 
platform  with  FIRe  and  video  imaging  hardware  enabling  the  capture  of  both  video  mosaic  and 
FIRe  data  simultaneously.  This  acquisition  of  100’s  to  1000’s  of  FIRe  measurements  during  a 
single  dive  survey  will  provide  much  greater  health  information  about  the  reef  as  a  whole  than  is 
capable  using  the  single  colony  approach  (Figure  A66.2).  The  dense  FIRe  measurements  and  the 
spatial  reference  data  available  from  the  high  resolution  mosaics  will  also  allow  users  to  create 
dense  probability  maps  of  reef  health  which  could  lead  to  the  identification  of  local  hotspots  of 
stress  along  affected  reefs  (Figure  A66.3).  Over  time,  these  maps  could  be  used  to  track  both 
visible  and  physiological  changes  in  reefs  at  a  landscape  level  and  help  determine  the  impacts  of 
reef  stressors  in  a  way  that  is  currently  inaccessible  to  modem  reef  scientists. 

A2.5. 7  NOAA  Restoration  Group 


In  May  of  2006  two  species  of  coral,  Acropora  palmata  and  Acropora 
cervicornis,  were  listed  as  “threatened”  on  the  endangered  species  list  due  to 
their  widespread  declines  throughout  the  Caribbean.  Under  the  Endangered 
Species  Act  (ESA),  16  USC  §§1531-1544  Federal  agencies  must  “insure  that 
any  action  authorized,  funded,  or  carried  out  by  such  agency  is  not  likely  to 
jeopardize  the  continued  existence  of  any  endangered  species  or  threatened 
species  or  result  in  the  destruction  or  adverse  modification  of  habitat  of  such  species.”  In 
accordance  with  this  Federal  protection,  governmental  stakeholders  are  responsible  for  the 
documentation  and  monitoring  of  these  threatened  species  with  respect  to  any  DoD  or  NOAA 
actions.  To  ensure  this  protection,  the  NOAA  restoration  group  in  association  with  the 
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University  of  Miami/RSMAS  has  begun  to  monitor  two  local  populations  of  Acropora  palmata 
in  the  Florida  Keys  using  landscape  video  technology. 

The  two  monitoring  sites  in  the  Upper  Florida  Keys  were  chosen  to  represent  different  stages  of 
A.  palmata  population  health.  Both  sites  were  marked  and  surveyed  in  June  2009.  Site  one  at 
South  Carysfort  Reef  is  a  shallow  site  (~2m  deep)  that  experienced  significant  declines  over 
recent  decades  but  is  now  showing  moderate  recovery  (Figure  A67).  Colonies  are  sparsely 
distributed  and  smaller  in  overall  size  than  Site  two.  Numerous  small  colonies  are  distributed 
throughout  the  site  representing  recent  cementation  and  growth  events. 


Figure  A67.  South  Carysfort  Reef,  Florida  Keys.  Large  dead  skeletons  and  mostly  sparse  small  colonies 
distributed  throughout  the  site  characterize  this  population  of  threatened  coral  species  Acropora  palmata. 


Site  two  at  Horseshoe  Reef  is  perhaps  the  largest  grouping  of  live  Acropora  palmata  left  in  the 
Florida  Keys  (Figure  A68).  This  stand  is  deeper  (~3m)  than  the  Carysfort  site  and  consists  of 
very  densely  packed  Acropora  palmata  colonies.  Colonies  at  this  site  appear  healthy  with  some 
evidence  of  fish  predation.  This  stand  is  believed  to  have  survived  much  of  the  catastrophic 
damage  to  their  population  as  seen  elsewhere  in  the  Florida  Keys  during  the  last  few  decades. 
Colonies  are  densely  packed  with  several  layers  of  healthy  A.  palmata  understory.  The  site  was 
surveyed  as  two  overlapping  sections  (both  ~400m2)  and  will  be  used  along  with  mosaics  at  site 
one  to  inventory,  map,  and  follow  these  stands  of  threatened  Acropora  palmata  through  time. 
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Figure  A68.  Horseshoe  Reef,  Florida  Keys.  This  population  of  Acropora  palmata  is  one  of  the  largest  and 
healthiest  stands  remaining  in  the  Florida  Keys  National  Marine  Sanctuary.  The  image  above  represents 
650m2  and  is  only  half  of  the  entire  mosaic  study  site. 


The  branching  growth  form  and  life-history  characteristics  of  A.  palmata  pose  a  considerable 
challenge  to  traditional  reef  monitoring  methods.  Acroporids  often  grow  in  shallow,  high-energy 
reef  areas  as  thickets  of  interlaced  colonies.  This  often  limits  our  ability  to  distinguish  individual 
colonies  and  makes  the  deployment  of  lines  and  quadrats  cumbersome  and  damaging  to  the 
resources  that  can  be  fragmented  easily.  Furthermore,  the  main  mode  of  reproduction  of 
Acroporid  species  is  through  branch  detachment  and  fragmentation  (Lirman,  2000),  which  can 
produce  very  quick  and  dynamic  changes  in  colony  size  and  abundance  that  are  not  easily 
tracked  through  conventional  diver-based  methods. 

Due  to  these  difficulties,  mapping  methodologies  that  cover  areas  larger  than  single  colonies  will 
be  beneficial  for  assessing  the  status  of  populations  of  these  threatened  species  and  their  recovery 
through  time.  The  use  of  video  mosaic  monitoring  technology  in  areas  of  threatened  A.  palmata 
and  A.  cervicornis  species  provides  a  non-invasive  and  large  area  method  of  recording  the 
current  status  of  these  species  in  a  given  area.  Individual  colonies  and  thickets  can  be  monitored 
through  time  without  the  need  for  tagging  of  individual  colonies  (Gleason  et  al.,  2007).  A 
landscape  mosaic  will  permanently  acquire  an  overall  detailed  view/image  of  a  given  reef  habitat 
and  its  benthic  components  including  ESA-listed  corals  Acropora  cervicornis  and  A.  palmata 
and  their  designated  critical  habitat.  These  mosaics  will  yield  information  on  the 
presence/absence  of  ESA-listed  corals  A.  cervicornis  and  A.  palmata,  as  well  as  accurate 
estimates  of  A.  cervicornis  and  A.  palmata  cover  or  density  per  sampled  area,  and  colony  size.  A 
succession  of  mosaics  over  time  will  enable  the  accurate  monitoring  of  the  dynamics  of  A. 
cervicornis  and  A.  palmata  at  a  given  site. 

A2.5.8  NOAA  Marine  Heritage 

The  National  Oceanic  and  Atmospheric  Administration  Office  of  National  Marine  Sanctuaries 
(NMS)  coordinates  efforts  to  document  national  maritime  heritage  resources.  Landscape  mosaics 
may  help  documentation  and  communication  efforts  for  many  of  the  underwater  archaeological 
sites  they  survey. 
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Trained  underwater  archaeologists  have  intensively  surveyed  only  a  relative  handful  of  high- 
priority  sites,  such  as  the  USS  Monitor.  For  such  high-priority  sites,  landscape  mosaics  can 
complement  traditional  methods  of  underwater  archaeology  by  helping  document  large-scale 
changes  to  the  site  over  time  and  facilitate  effective  communication,  for  example  in  educational 
displays  or  web  sites  describing  the  site.  NOAA  NMS  does  not  have  the  resources  to  conduct 
intensive  excavations  or  create  base  maps  for  the  vast  majority  of  underwater  heritage  sites, 
however.  For  these  numerous  lower-priority  sites,  at  which  imagery  is  likely  to  be  the  only 
source  of  data,  landscape  mosaics  can  provide  crucial  documentation  of  the  artifacts  and  site 
condition. 

We  conducted  a  pilot  effort  to  evaluate  whether  the  landscape  mosaic  software  could  contribute 
to  the  NMS  mosaic  efforts.  The  schooner  Kyle  Spangler  was  chosen  as  a  test  site  because  NMS 
archaeologists  had  previously  created  both  a  detailed  archaeological  base  map  and  a  hand-made 
landscape  mosaic  (http://thunderbay.noaa.gov/shipwrecks/spangler.html).  The  resulting  mosaic 
produced  from  our  software  (Figure  A69.A)  resembles  the  hand-made  mosaic  made  by  NMS 
archaeologists  (Figure  A69.B).  The  two  primary  differences  between  the  mosaics  were,  first,  that 
the  main  (aft)  mast  has  been  cropped  out  of  the  automated  mosaic,  and,  second,  that  the  scale  is 
not  constant  in  the  automated  mosaic.  Differences  in  data  acquisition  between  the  method 
normally  employed  by  the  RSMAS  team  for  coral  reef  surveys  and  the  method  used  for  the 
Spangler  survey  explain  the  differences  in  the  resulting  mosaics. 

The  cropped  mast  is  due  to  an  extreme  violation  of  one  of  the  assumptions  used  in  the  automated 
mosaic  algorithm,  namely  that  the  maximum  relief  in  the  scene  is  small  relative  to  the  altitude  of 
the  camera  above  the  seabed.  Although  there  is  no  hard-and-fast  rule  about  what  constitutes 
“small”  relief,  the  normal  rule  of  thumb  used  by  the  RSMAS  team  when  acquiring  imagery  for 
coral  reef  surveys  is  to  place  the  camera  about  1.5  to  2  times  as  high  as  the  highest  relief  object 
in  the  scene.  For  example,  if  there  were  aim  high  coral  head  in  the  area  surveyed  we  would 
normally  acquire  data  1.5  to  2  m  above  the  bottom.  The  Spangler  images  containing  the  masts 
obviously  violate  this  “small  relief’  assumption  as  the  camera  was  actually  below  the  tops  of  the 
masts.  Fortunately,  most  of  the  images  in  the  dataset  did  not  have  such  extreme  relief.  The 
blending  scheme  used  by  the  automated  mosaic  method  has  cropped  out  the  mast  because  it  was 
designed  to  reduce  the  deleterious  effects  of  parallax  on  the  presentation  of  the  final  mosaic.  The 
reason  that  it  removed  only  the  main  mast  and  not  the  forward  mast  is  probably  due  to  the 
number  of  images  available.  Near  the  ends  of  the  ship  there  were  probably  fewer  potential 
images  that  could  have  been  placed  in  the  mosaic,  so  cutting  out  the  forward  mast  might  have 
left  a  hole  in  the  mosaic. 

The  scale  variation  in  the  automated  mosaic,  visible  as  the  front  portion  of  the  ship  appearing 
wider  than  the  rear,  was  caused  by  a  different  approach  to  data  acquisition.  The  automated 
algorithm  relies  on  substantial  overlap  in  both  along-track  and  across-track  directions  and 
performs  best  when  “tie-lines”  (i.e.  multiple  passes  over  the  same  portion  of  the  scene)  are 
available.  NMS  acquired  the  data,  however,  as  just  a  single  pass  down  one  side  of  the  vessel  and 
a  second  pass  up  the  other  side.  This  approach  minimizes  time  in  the  water  and  gave  sufficient 
images  for  a  manual  match,  but  was  sub-optimal  for  the  automated  algorithm.  The  automated 
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mosaic  of  the  Spangler  was  created  from  only  24  frames,  whereas  the  coral  reef  mosaics 
typically  use  between  800  -  5000  frames  to  ensure  sufficient  overlap. 

The  experience  with  the  Spangler  mosaic  suggests  that  automated  landscape  mosaics  have  the 
potential  to  assist  NMS  marine  heritage  resource  inventory  personnel.  Results  indicate  two 
avenues  to  pursue  to  improve  future  results.  First,  we  should  coordinate  with  the  divers  acquiring 
data  to  make  some  small  adjustments  to  the  survey  methodology  to  increase  overlap  and  add  tie¬ 
lines.  Second,  modifications  to  the  algorithm  might  be  possible  to  detect  and  retain  areas  of  high 
parallax,  as  opposed  to  the  current  method  of  detecting  and  discarding  areas  of  high  parallax. 
Modifying  the  algorithm  in  this  way  would  require  additional  research  effort  and  would  probably 
degrade  results  over  reefs,  which  could  be  overcome  by  reverting  to  the  current  version  of  the 
algorithm  for  reef  work  and  using  the  modified  version  only  when  necessary. 


Figure  A69.  Landscape  mosaics  of  the  Kyle  Spangler.  Mosaic  created  using  the  automated  procedures 
developed  in  this  project  (Top).  Mosaic  created  by  NMS  by  hand-matching  each  image  (Bottom). 


A2.5.9  Florida  Fish  and  Wildlife  Conservation  Commission 


The  Coral  Reef  Evaluation  and  Monitoring  Project  (CREMP,  funded  by 
EPA,  NOAA,  and  the  State  of  Florida)  is  an  annual  survey  of  over  40 
permanent  monitoring  sites  throughout  the  Florida  Reef  Tract.  As  part  of 
their  coral  survey  methodology,  their  team  uses  underwater  video 
transects  that  are  subsequently  subsampled  by  extracting  individual 
video  frames.  These  video  frames  are  analyzed  using  point-count 
methods  to  determine  percent  cover  of  corals.  This  monitoring  program 
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represents  one  of  the  longest  running  programs  in  the  Florida  Keys  and  was  one  of  the  original 
programs  to  implement  video-based  reef  monitoring  into  their  survey  methodologies. 

In  November  2009,  a  pilot  collaboration  effort  was  conducted  between  the  University  of 
Miami  and  CREMP  to  co-survey  two  sites  (consisting  of  two  stations  each)  using  CREMP 
standard  methods  as  well  as  UM’s  Video  Mosaic  method.  Both  data  sets,  collected  at  the  same 
time,  will  be  analyzed  and  the  metrics  of  coral  condition  obtained  will  be  compared  between 
programs  to  determine  whether  video  mosaics  can:  1)  provide  comparable  estimates  of  coral 
cover,  and  2)  expand  the  number  of  indicators  of  coral  condition  that  can  be  extracted  from  each 
site  using  video-based  survey  methodologies.  In  addition  the  collaboration  hopes  to  utilize 
historical  video  data  to  create  a  10+  year  comparison  of  video  mosaic  sites  in  the  Florida  Keys. 


Figure  A70.  Video  mosaic  of  2mx20mtransect  at  Sombrero  Reef,  Florida  Keys. 


A2. 5. 10  Potential  Department  of  Defense  Collaborations/Users 

The  following  report  was  compiled  by  Lorri  Schwartz  and  colleagues,  NAVFACHQ. 

DEPARTMENT  OF  DEFENSE  NEEDS  FOR  MAPPING,  INVENTORY,  AND  ASSESSMENT 
OF  BENTHIC  MARINE  COMMUNITIES 

The  Department  of  Defense  (DoD)  needs  to  inventory,  identify,  document  and  assess  benthic 
reef  communities  and  other  benthic  habitats  in  order  to  have  baseline  information  to  comply  with 
regulations  and  resource  management  requirements  in  proximity  to  installations  and  operational 
areas.  DoD  utilizes  tools  such  as  Habitat  Equivalency  Analysis  for  performing  analysis  of 
potential  impacts  for  construction  activities.  Additionally,  DoD  needs  to  conduct  monitoring  of 
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benthic  habitats  in  order  to  fulfill  NEPA  or  permit  mitigation,  Trustee  obligations  or  other 
conservation  commitments.  The  benthic  reef  community  includes  corals,  algae,  and  other  sessile 
and  mobile  invertebrates  and  associated  substrates. 

Technologies  fulfilling  these  needs  will  provide  operators  and  natural  resources  personnel  with 
comprehensive  knowledge  of  benthic  habitats  and  coral  reef  communities  under  DoD  purview. 
This  information  is  necessary  for  operational  and  environmental  planning  and  provides  decision¬ 
makers  with  crucial  information  needed  to  maintain  compliance  with  statutes,  regulations,  and 
executive  orders  directly  related  to  operations  conducted  in  benthic  areas,  including: 

■  Clean  Water  Act  (33  USC  §  125 1  et  seq.) 

■  Coastal  Zone  Management  Act  of  1972(16  USC  §§1451-1465) 

■  Comprehensive  Environmental  Response,  Compensation,  and  Liability  Act  (42  USC 
Chapter  103) 

■  Coral  Reef  Conservation  Act  of  2000  (16  USC  §6401  et  seq) 

■  Magnuson-Stevens  Fisheries  Conservation  and  Management  Act  (16  USC  §§1801-1882) 

■  Marine  Protection,  Research  and  Sanctuaries  Act  (16  USC  §  §  1 43 1-1 445a) 

■  National  Environmental  Policy  Act  as  amended  (42  USC  §§4321-4347) 

■  Oil  Pollution  Act  of  1990,  33  USC  §2701 

■  Rivers  and  Harbors  Act  (33  USC  §403) 

■  Sikes  Act  Improvement  Act  of  1997  (16  USC  §670a-o) 

■  Executive  Order  13089,  Coral  Reef  Protection 

■  Executive  Order  12114,  Environmental  Effect  Abroad 

■  Executive  Order  12777,  Oil  Pollution  Act  Implementation 

■  Executive  Order  13158,  Marine  Protected  Areas 

Obtaining  baseline  data  is  an  important  element  not  only  for  Federal  coastal  management  of 
protected  resources  but  to  provide  a  foundation  for  environmental  documentation  necessary  to 
conduct  operations.  Such  documentation  requires  the  assessment  of  environmental  conditions 
prior  to  any  incidents  possibly  resulting  in  damage  to  or  loss  of  habitat.  Successful  and  legally 
defensibly  documentation  requires  the  assessment  of  environmental  conditions  prior  to 
conducting  operations  and  implementation  of  mitigation  measures.  Assessment  information  is 
also  necessary  in  resolving  Federal  trustee  matters  related  to  damage  assessments.  Legally 
defensible  data  is  necessary  to  communicate  and  negotiate  all  regulatory  actions  in  the  marine 
environment. 

■  Efficient  assessment  of  benthic  habitats  to  support  routine  activity  planning 

■  Reduced  time  and  expense  for  data  collection 

■  Reasonable  operator  experience  and  dive  time  requirements 

■  Experts  spend  more  time  in  lab  analyzing  data  than  in  field  collecting  data 

■  Applicable  in  a  wide  range  of  locations  (see  military  facility  table  below) 

■  Support  day  or  night  data  collection  as  required 

■  Data  quality  to  support  compliance  requirements 

■  Quantitatively  and  qualitatively  characterize  the  diversity,  abundance,  temporal 
variation  and  spatial  distribution  of  corals,  algae  and  other  invertebrates 

■  Support  Habitat  Equivalency  Analysis  tool  and  NEPA  analyses,  as  well  as  permit 
and  mitigation  compliance 
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■  Provide  a  common  monitoring  protocol  for  the  benthic  community  was 
formulated  with  regards  to  location  and  frequency  of  surveys 

■  Robust,  reliable  and  legally  defensible 

■  Locate  survey  start  and  end  points  located  using  Global  Position  System  (GPS) 
sensors 

■  Provide  data/image  archival  capability  and  data  compatibility  with  existing 
software  including  military  GIS  applications  (EIMS  and  PMAP) 

■  Facilitate  interoperability  between  DoD  components  and  cooperation  with  other  Federal 
and  State  agencies  for  compliance  and  stewardship  efforts. 

■  Mutual  benefit  to  use  same  tools 

■  Cost  savings  to  share  the  same  data  for  regulatory  needs. 

■  Low  cost,  high  benefit,  ease  of  deployment  will  allow  expanded  benthic  habitat 
assessment  and  monitoring 

■  Potential  to  leverage  research  needs 

Table  A13.  Military  Facilities  with  Adjacent  Coral  Reef  Resources 


Branch 

Facility  Name 

Location 

Air  Force 

Anderson  Air  Force  Base 

Guam 

Air  Force 

Cape  Canaveral  Air  Force  Station 

Florida 

Air  Force 

Eglin  Gulf  Test  and  Training  Range  (EGTTR) 

Florida 

Air  Force 

Hickam  Air  Force  Base 

Hawaii 

Air  Force 

Tyndall  AFB 

Florida 

Air  Force 

Bellows  Air  Force  Station 

Hawaii 

Air  Force 

Patrick  Air  Force  Base 

Florida 

Air  Force 

Wake  Atoll  (Wake  Island) 

US  Territory 

Air  Force/ 
Navy 

Eglin  AFB 

Florida 

Army 

Fort  Buchanan 

Puerto  Rico 

Army 

Fort  Shafter 

Hawaii 

Army 

Johnston  Atoll  Chemical  Agent  Disposal  System 
Facility  (JACAD) 

US  Territory 

Army 

Kwajelein  Atoll,  Reagan  Test  Site,  Marshall  Islands 

US  Territory 

Army 

Pohakuloa  Training  Area 

Hawaii 

Army 

Schofield  Barracks 

Hawaii 

Army 

Tripler  Army  Medical  Center 

Hawaii 

Marine 

Corps 

Marine  Corps  Base  Hawaii 

Hawaii 

Marine 

Corps 

Marine  Corps  Base  Hawaii  Ranges 

Hawaii 

Navy 

Andros  Island,  AUTEC 

Bahamas 

Navy 

Awase  Transmitter  Site,  Okinawa 

Japan 

Navy 

Barbers  Point  Family  Housing  and  Support 

Hawaii 

Navy 

Diego  Garcia  Navy  Support  Facility 

BIOT 

Navy 

Diego  Garcia  Range  Complex 

BIOT 
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Branch 

Facility  Name 

Location 

Navy 

Farallon  De  Madinilla  (FDR) 

CNMI 

Navy 

Ford  Island  Naval  Station  Annex 

Hawaii 

Navy 

Guam  Naval  Activities 

Guam 

Navy 

Guantanamo  Bay  Naval  Station 

Cuba 

Navy 

Guantanamo  Complex 

Cuba 

Navy 

Gulf  of  Mexico  Training  Area 

Florida 

Navy 

Hawaiian  Range  Complex 

Hawaii 

Navy 

Japan  Range  Complex 

Japan 

Navy 

Key  West  Range  Complex 

Hawaii 

Navy 

Key  West  Naval  Air  Station 

Florida 

Navy 

Marianas  Range  Complex 

CNMI 

Navy 

NAMFI  Complex 

Mediterranean 

Navy 

NASD,  EMA  &  AFWTF 

Puerto  Rico 

Navy 

Naval  Supply  Center  Red  Hill 

Hawaii 

Navy 

Okinawa  Naval  Activities 

Japan 

Navy 

Okinawa  Complex 

Japan 

Navy 

Pachino  Complex 

Mediterranean 

Navy 

Pacific  Missile  Range  Facility  (PMRF)  Barking 
Sands,  Kauai 

Hawaii 

Navy 

Panama  City  Coastal  Systems  Center 

Florida 

Navy 

Pearl  Harbor  Naval  Station 

Hawaii 

Navy 

Pensacola,  Naval  Air  Station 

Florida 

Navy 

Tinian  Island,  Military  Leased  Areas 

CNMI 

Navy 

White  Beach  Naval  Facility,  Okinawa 

Japan 

A2.5.11  Landscape  Mosaics  website 

In  2008  a  website  detailing  information  on  underwater  landscape  mosaics  was  constructed  and 
put  on-line  for  users  to  have  free  access  to  basic  information  on  mosaic  creation,  current  coral 
monitoring  projects  using  this  technology,  and  contact  information  for  any  interested  parties. 
The  website  is  continually  being  updated  to  include  more  information  and  better  imagery  as  they 
become  available.  Lists  of  our  publications  and  PDF's  of  many  of  our  posters  are  also  available 
here.  The  web  address  is: 
www.rsmas.miami.edu/groups/reidlab 
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Figure  A71.  Screen-capture  of  landscape  mosaic  website  homepage  www.rsmas.miami.edu/groups/reidlab. 


A2.6  Software  deliverables 

A2.6.1  Landscape  (2D)  Mosaic  Software 

A  significant  portion  of  our  efforts  in  the  final  two  years  of  the  project  focused  on  the 
development  of  user-friendly  software  for  use  by  external  users  for  the  creation  of  2D  video 
mosaics.  As  such,  a  set  of  2D  mosaicing  programs  has  been  written  to  run  under  the 
commercially  available  Matlab  computation  platform.  Additionally  a  detailed  user  manual  was 
produced  to  cover  the  use  of  all  2D  mosaic  creation  programs  and  is  provided  in  Appendix  3.  A 
brief  introduction  to  these  programs  is  given  below. 

The  Mosaic  Creation  program  allows  a  user  to  create  a  2D  video  mosaic  from  an  “AVI”  media 
file.  The  program  and  interface  consists  of  six  main  modules:  1)  Frame  Extraction,  2)  Sun 
Flicker  Correction,  3)  Global  Matching,  4)  Video  and  Still  Registration,  5)  Global  Match 
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Inspection  and  6)  Mosaic  Rendering.  When  the  Mosaic  Creation  toolbox  is  executed,  a  toolbox 
menu  will  appear  showing  the  main  steps  of  mosaic  creation  as  the  user’s  options  (Figure  A72). 


Figure  A72.  Main  Mosiacing  Program  GUI 

A2. 6.1.1  Frame  Extraction 


As  a  first  step,  users  extract  individual  video  frames  from  an  input  “AVI”  media  file, 


compensate  for  camera  lens  distortion  and  remove  interlaced  scan  lines.  This  is  accomplished  by 
clicking  on  the  “Correct  AVI  Frames”  option  in  the  toolbox  menu  (shown  above).  This  then 
opens  a  new  “Frame  Extractor”  interface  (Figure  A73). 


s  d_co  r  recta  vif  rames 


Figure  A73.  A  screen  capture  of  the  video  frame  extraction  GUI. 

The  user  can  correct  the  color  balancing  and  enhance  the  contrast,  by  providing  black  and  white 
references.  The  output  is  a  set  of  undistorted  individual  video  frames  that  are  the  basis  for  the 
next  steps  in  the  mosaic  creation  process. 
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A2. 6.1.2  Sunflickering  Removal 

The  second  module  is  targeted  at  removing  sunflickering  and  is  optional.  The  processing  is  done 
in  two  steps:  (1)  Sequential  matching  and  (2)  sunflicker  filtering.  The  first  step  of  sequential 
matching  finds  the  registration  parameters  among  time-consecutive  images.  In  this  step, 
normalized  cross  correlation  is  used  as  a  similarity  measure  between  images,  since  it  outperforms 
the  Scale  Invariant  Feature  Transform  (SIFT)  method  for  images  heavily  affected  by  refracted 
sunlight.  The  SIFT  method  is  used  in  our  mosaicing  software  for  matching  images  under  normal 
illumination  conditions.  The  second  step  of  sunflicker  filtering  step  uses  the  registration 
parameters  to  estimate  and  remove  the  instantaneous  illumination  field  caused  by  the  sunlight. 

The  user  interface  is  shown  in  Figure  A74.  The  separation  of  the  processing  in  two  steps  allows, 
if  needed,  for  re-running  filtering  step  with  different  parameters  without  having  to  re-run  the 
matching,  which  is  the  most  time  consuming  part  of  the  method. 


Figure  A74.  Graphic  user  interface  for  the  sunflicker  removal  module. 


A2. 6.1.3  Global  Matching 


The  third  module,  called  Global  Matcher  performs  three  main  tasks: 

•  Image  selection:  computes  image  features,  performs  sequential  matching  and  removes 
frames  that  are  found  to  have  very  large  overlap 

•  Global  matching:  computes  the  fast  similarity  measure  among  all  images  and  performs 
matching  for  all  pairs  of  high  similarity 
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•  Global  optimization:  finds  the  best  displacement  for  all  images,  using  the  result  of  the 
global  matching. 

The  Global  Matcher  interface  (Figure  A75)  allows  for  setting  all  the  relevant  mosaicing 
parameters,  such  as  the  range  of  image  to  be  matched,  and  the  thresholds  for  image  selection  and 
matching.  These  thresholds  can  be  adjusted  to  improve  the  matching  performance  for 
particularly  difficult  image  sets  (for  example  containing  strong  topography).  Upon  finishing,  data 
files  are  created  with  all  the  information  needed  to  blend  the  mosaic. 


Figure  A75.  The  global  matching  interface. 

A2.6.1.4  Video  and  Still  Registration 

The  forth  module  allows  for  registering  high  definition  stills  against  the  video  frames  that  were 
used  in  a  run  of  the  Global  Matching  module,  using  the  user  interface  shown  in  Figure  A76. 
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Figure  A76.  The  video  frame  and  still  image  registration  interface. 

A2.6.1.5  Global  Match  Inspection 

The  fifth  module  is  the  Global  Match  Inspector  (Figure  All).  This  optional  module  allows  the 
user  to  inspect  the  results  of  the  global  matching.  More  specifically,  the  user  will  be  able  to  view 
individual  pairs  of  matched  images  (with  the  corresponding  feature  points  overlaid),  manually 
add  or  clear  global  matches  to  correct  optimization,  as  well  as  create  new  optimization  files  for 
the  rendering  process. 


Figure  All.  The  match  inspector  GUI. 
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A2. 6.1.6  Mosaic  Rendering  with  Improved  Blending 


The  sixth  module  is  the  Mosaic  Rendering  (Figure  A78).  It  renders  a  complete  mosaic  using  the 
results  of  the  global  matching  and  optimization. 


Figure  A78.  Mosaic  Rendering  GUI. 


The  user  may  select  an  image  subset  to  render  by  clicking  on  a  section  of  the  global  matching 
table.  This  table  is  useful  to  check  the  parts  of  the  sequence  that  contain  loops,  which  typically 
correspond  to  the  intended  survey  area.  The  user  may  also  perform  a  variety  of  image  blending 
techniques:  Closest,  Graph-Cut  and/or  Gradient  Blending.  The  final  outcome  is  a  “PNG”  format 
picture  file  created  at  a  desired  resolution. 


A2. 6.1.7  Heading  Integration  Module 

The  heading  integration  module  is  intended  to  be  used  with  the  hardware  developed  for  the 
second-generation  mosaics.  This  module  processes  the  audio  file  that  contains  the  raw  heading 
information  (recorded  in  the  video  camera  audio  channel)  and  converts  the  information  into 
heading  angles  (GUI  shown  in  Figure  A79). 
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Figure  A79:  Heading  integration  module. 

A2. 6.1.8  Additional  Features 


A  few  extra  features  have  been  added  to  complement  the  mosaic  creation  deliverable  and  allow 
some  ecological  analysis  of  the  final  mosaic.  The  first  is  the  “Point  and  Click”  feature,  allowing 
the  user  to  select  a  desired  point/area  on  the  mosaic  and  view  the  corresponding  full  resolution 
video  frames.  Calling  the  “point_click”  function  in  the  Matlab  command  line  will  display  the 
following  interface  (Figure  A80). 


Figure  A80:  Basic  point  click  GUI. 


With  the  pathname  of  the  mosaic  and  the  corresponding  registration  file,  the  user  would  simply 
click  the  “Select  a  Point”  button  and  select  a  point/area  in  the  mosaic  of  interest.  The  closest 
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frame  to  the  desired  area  will  be  displayed  next  to  the  mosaic  as  well  as  a  feature  box  showing 
the  frame’s  place  in  the  mosaic,  as  seen  below.  This  provides  users  with  an  un-manipulated  view 
of  the  area  or  point  of  interest.  Often  this  provides  a  clearer  image  than  the  2D  mosaic  and  can 
provide  additional  health  information  (Figure  A81). 


Figure  A81.  Basic  Point  Click  GUI  showing  a  selected  video  frame. 


The  second  ecological  analysis  feature  is  the  “Mosaic  Info”  function.  Calling  “mosaic_info”  in 
the  Matlab  command  line  will  call  the  following  interface  to  the  user  (Figure  A82) 


Figure  A82.  Mosaic  Info  GUI. 
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A  total  description  of  the  mosaic  is  displayed  to  the  right,  including  information  of  the  size  of  the 
mosaic  (in  Pixels)  and  the  area  covered  during  the  mosaic  survey  (in  meters).  The  user  can 
determine  the  distance  (in  meters)  between  two  points  within  the  mosaic  by  clicking  the  “Find 
Distance/Marker”  button.  This  has  been  used  to  assess  the  size  of  coral  colonies  within  a  video 
mosaic  as  well  as  spatial  patterns  of  coral  distributions.  After  selecting  desired  points  of  interest 
in  the  area,  the  user  is  given  the  option  to  save  a  copy  of  the  mosaic  with  the  points  marked  on  it 
for  reference,  as  well  as  an  excel/csv  output  file  that  contains  the  distances  of  the  markers  to  each 
boundary  comer  (Figure  A83). 


Figure  A83.  A  mosaic  with  distance  markers  created  by  the  mosaicinfo  GUI. 


For  further  information  on  the  Mosaic  Creation  steps  and  processes,  please  see  Appendix  3, 
“Mosaic  Creation  Manual”. 


A2.6.2  Ecological  Analysis  Module 

An  objective  of  the  landscape  mosaicing  system  is  to  provide  end  users  with  improved 
monitoring  products  that  provide  important  information  on  reef  health.  One  index  of  reef  health 
commonly  used  by  monitoring  and  scientific  purposes  is  that  of  percent  cover.  When  using 
benthic  imagery  to  extract  this  index,  users  typically  upload  images  to  point  count  software  (e.g. 
CPCe,  Kohler  and  Gill,  2006)  that  displays  a  number  of  random  points  within  the  image  that  are 
then  assessed  for  benthic  composition.  Landscape  mosaics  have  been  used  with  pre-existing 
software  (namely  CPCe)  to  extract  indices  of  percent  cover  directly  from  mosaics  or  from 
portions  of  mosaic  images.  However,  due  to  the  limited  resolution  of  first-generation  mosaic 
products  percent  cover  indices  were  often  limited  to  major  benthic  categories  such  as  sand, 
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macroalgae,  coral,  soft  coral  and  zoanthids.  Thus  information  on  species-level  cover  or  different 
macroalgal  groups  was  not  available  using  first-generation  products. 

The  creation  of  the  second-generation  of  landscape  mosaics  provides  a  product  with  thousands  of 
incorporated  still  images.  These  high  resolution  still  images  provide  sub-millimeter  resolution  for 
end-users,  species  level  identification  capabilities  for  most  corals,  and  genera  level  for  most 
macroalgae.  Because  these  improvements  in  resolution  are  substantial,  it  is  important  to  be  able 
to  utilize  the  information  within  the  georeferenced  images  in  point  counting  analyses.  Up  until 
this  point  the  only  way  to  use  the  georeferenced  still  images  within  common  point  count  analysis 
software  was  to  identify  the  area  in  which  analysis  is  needed  and  using  Matlab  attempt  to  click 
on  the  area  where  the  point  was  for  identification  purposes.  To  make  this  process  easier,  and  to 
better  incorporate  second-generation  mosaic  products  with  existing  image  analysis  software,  we 
have  created  an  automated  extraction  program  that  uses  CPCe  point  count  information  to  not 
only  identify  benthic  points  within  a  mosaic  image  but  also  within  the  closest  corresponding  still 
image. 

CPCe  Points  Extraction  module 

The  Coral  Point  Count  program  is  a  Windows-based  program  that  provides  a  tool  for  the 
determination  of  coral  cover  using  transect  photographs  (in  this  case  a  rendered  mosaic).  Note: 
CPCe  program  developed  by  Kevin  E.  Kohler  (httpC/www.  nova,  edu/ocean/cpce/) 

To  use  the  CPCe  points  extraction  module  the  user  will  first  open  a  mosaic  in  the  CPCe  program, 
select  an  area  of  interest  and  establish  the  number  of  random  points  that  will  be  created  (Figure 
A84).  Each  point  is  given  a  unique  name  (usually  letters  A,  B,  C  etc)  that  is  used  to  identify  the 
substrate  under  each  set  of  crosshairs. 
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Figure  A84.  Screen-capture  of  a  mosaic  image  being  used  with  CPCe  point  count  program.  The  user  has 
selected  an  area  of  interest  (yellow  box)  and  20  random  points  within  the  image  (points  shown  in  green).  The 
user  can  then  save  a  .cpc  file  with  the  locations  of  each  point  within  the  image  that  is  then  used  to  extract  the 
corresponding  still  images. 

Once  the  points  have  been  established  on  the  mosaic  file,  the  user  will  save  the  set  of  points  as  a 
*.cpc  file.  This  creates  a  location  fde  for  each  of  the  points  created  by  the  program. 

After  the  CPC  file  has  been  created  for  the  desired  mosaic,  users  can  click  the  ‘Extract  Frames 
from  CPC  file’  button  on  the  “Point  and  Click  Video  and  Stills”  GUI.  Matlab  will  prompt  the 
user  to  find  and  select  the  appropriate  *.cpc  file  and  choose  an  output  folder  in  which  the 
extracted  images  will  be  saved  to.  This  feature  will  proceed  to  read  all  the  random  points  from 
the  selected  .cpc  file  and  extract  the  closest  video/still  frame  for  each  point.  The  exact  location  of 
the  random  point  will  be  computed  for  the  still  frame  and  a  cross-hair  corresponding  to  the 
random  point  will  be  placed  on  the  designated  still  image  (Figure  A85).  All  still  images  needed 
for  this  analysis  will  be  extracted  to  their  own  folder  and  renamed  with  the  name  of  the  random 
point  designated  by  CPCe.  These  images  can  then  be  analyzed  in  CPCe  or  any  other  image 
viewer.  The  result  of  this  process  is  the  ability  to  incorporate  second-generation  mosaic  products 
into  commonly  used  image  analysis  software  without  extensive  user  intervention. 
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Figure  A85.  Extracted  still  image  of  point  I.  The  location  of  the  random  point  is  computed  on  the  extracted 
frame  and  displayed  with  the  blue  crosshairs  shown  in  the  bottom  right  of  the  image.  Images  can  be  analyzed 
using  CPCe  or  any  other  image  viewing  program. 

A2.6.3  External  viewer 

The  external  viewer  allows  the  user  to  view  a  specific  area  in  a  mosaic  and  retrieve  the  closest 
original  frame  and  high-resolution  stills.  This  viewer  runs  outside  of  the  Matlab  environment, 
and  can  be  easily  distributed  as  a  standalone  windows  applications. 

The  input  required  by  the  external  viewer  is  generated  using  a  simple  interface  shown  below. 
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Figure  A86.  Screenshot  of  GUI  that  creates  the  file  used  for  the  external  point  and  click  viewer. 


When  open,  the  external  mosaic  viewer  allows  the  user  to  click  on  the  mosaic  to  retrieve 
individual  video  frame  and  high  resolution  still. 


Figure  A87.  External  point  click  viewer  interface. 
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Figure  A88.  Result  of  clicking  on  mosaic  on  interface.  The  two  screens  brought  up  are  the  still  image  (on  the 
left)  and  the  video  frame  (on  the  right). 


A3.  Conclusions  and  Implications  for  Future  Research/Implementation: 

SERDP  project  SI  1333  has  developed  and  assessed  the  use  of  a  novel  technology-underwater, 
landscape  (2D)  image  mosaics-  for  use  in  coral  reef  mapping  and  monitoring.  A  major  goal  in 
this  project  was  to  determine  whether  landscape  mosaics  address  critical  limitations  of 
traditional,  diver-based  assessment  (considered  to  be  the  ‘gold  standard’  in  reef  monitoring), 
while  simultaneously  retaining  the  strengths  of  the  diver-based  approach  (Table  A14).  Our 
overall  assessment  is  that  mosaic  technology  accomplishes  this  role,  thereby  advancing  state-of- 
the-art  capabilities  for  reef  mapping  and  monitoring  in  a  variety  of  important  ways. 

Landscape  mosaics  are  automatically  created  from  numerous,  overlapping,  nadir-view, 
underwater  images  acquired  within  a  few  meters  of  the  seabed.  The  algorithms  used  to  create  the 
mosaics  evolved  over  the  duration  of  the  project  (Section  A2.1).  Initial  mosaicing  capability  used 
algorithms  developed  by  Neghadaripour  and  Gracias  prior  to  the  start  of  the  project  to  stitch 
together  frames  grabbed  from  a  video  camera.  Refinements  and  enhancements  to  the  Gracias 
algorithm  as  part  of  SI  1333  during  2003-2007  created  what  we  refer  to  as  our  ‘first-generation’ 
mosaicing  algorithm.  The  first-generation  algorithm  improved  mosaicing  capability  to  the  point 
where  the  hardware  used  for  data  acquisition  became  the  limiting  factor  for  mosaic  quality. 
During  2007-2009,  improvements  were  made  to  address  these  limitations  and  create  ‘second 
generation’  mosaics.  These  improvements  involved  integration  of  a  high  resolution  still  camera 
and  a  heading  sensor  with  the  video  acquisition  system.  The  integration  provided  increased 
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spatial  resolution  and  improved  performance  in  areas  with  high  topographic  relief.  In  addition, 
the  mosaicing  algorithm  was  upgraded  to  accommodate  the  additional  data.  Underwater 
landscape  mosaics  created  using  our  ‘second  generation’  system  cover  areas  of  up  to  400  m2 
with  sub-millimeter  pixel  resolution. 
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Table  A14:  Summary  of  advancements  to  the  state-of-the-art  in  coral  reef  monitoring  techniques. 
Capabilities  and  characteristics  of  various  techniques  are  rated  as  ‘excellent’  (green),  indicating  a  desired 
level  of  performance;  ‘adequate’  (yellow)  for  an  acceptable  level  of  performance;  and  ‘poor’  (purple), 
indicating  unsatisfactory  performance.  See  text  for  details. 


Performance  of  the  mosaicing  algorithms  improved  dramatically  throughout  the  course  of  SI 
1333  as  the  algorithm  was  refined  and  upgraded  (Section  A2.1.2).  Landscape  mosaics  created  at 
the  start  of  the  project  (i.e.  ‘initial  capability’  or  arrow  A  Table  A14)  addressed  the  limitations  of 
diver-based  surveys  and  resulted  in  improved  capabilities  relative  to  other  reef-monitoring 
image-based  techniques,  but  performed  poorly  in  terms  of  analysis  of  benthic  indicators.  ‘First 
generation’  mosaics,  produced  with  our  initial  upgraded  algorithm,  resulted  in  improvements  in 
analysis  capability,  such  that  traditional  benthic  indicators  could  be  extracted  from  the  mosaics 
with  a  degree  of  detail  that  was  adequate  for  many  applications  (arrow  B,  Table  A14).  Despite 
improvement  relative  to  the  initial  capability,  resolution  was  insufficient  for  detailed  benthic 
analysis.  Incorporation  of  a  still  camera  in  the  data  acquisition  system,  to  generate  ‘second- 
generation  landscape  mosaics’,  resulted  in  performance  that  was  comparable  to  the  ‘gold 
standard’  of  diver-based  transects  (arrow  C,  Table  A14).  The  increases  in  capabilities  shown  in 
Table  A14  are  summarized  in  the  following  paragraphs  (for  further  details,  see  Section  A2.3.3). 


124 


RC-1333  Final  Report  to  SERDP  2010 


R.P.Reid 


An  important  component  of  project  SI  1333  was  to  assess  whether  landscape  mosaics  retained 
the  strengths  of  diver-based  coral  reef  assessment.  Five  classic  indicators  of  reef  condition  were 
considered  in  addressing  this  question  (Table  A 14):  percent  benthic  cover,  diversity  index, 
juvenile  coral  density,  disease/bleaching/partial  mortality,  and  coral  colony  size.  Benthic 
indicators  extracted  from  our  initial  mosaics  compared  poorly  with  diver-based  measurements 
(see  ‘initial  capability’,  Table  A14).  Refinements  and  enhancements  performed  under  SI  1333  to 
create  the  ‘first-generation  mosaic  algorithm’  (Sections  A5.1.1,  5.1.2)  improved  the  ability  to 
extract  benthic  indicators  of  reef  condition  (see  ‘first  generation  mosaics’  Table  A14).  The 
percent  cover  of  corals  and  major  reef  taxa,  diversity  indices,  and  bleaching/disease/mortality 
metrics  analyzed  from  first-generation  mosaics  were  not  significantly  different  from  those 
analyzed  by  divers  as  long  as  analysis  was  restricted  to  major  groups  of  biotic  taxa  (e.g.  corals, 
macroalgae,  sand,  sponges;  Lirman  et  al.  2007).  As  coarse  levels  of  classification  are  acceptable 
for  many  reef  monitoring  purposes,  first-generation  performance  was  deemed  ‘adequate’  (Table 
A14).  With  the  higher  resolution  available  from  the  second-generation  mosaics,  analysis  was 
further  improved  such  that  species-level  identification  was  possible  (see  ‘second  generation 
mosaics’,  Table  A14).  As  a  result,  second-generation  performance  was  considered  ‘excellent’ 
(i.e.  on  a  par  with  divers;  Table  A14).  Sizes  of  coral  colonies  measured  from  both  first  and 
second-generation  mosaics  did  not  significantly  differ  from  diver-based  measurements,  and  thus 
mosaic  measurements  were  assessed  as  ‘excellent’  (Table  A14;  Lirman  et  al.  2007). 

One  category  where  ecologic  analysis  of  even  ‘second  generation  mosaics’  was  deemed 
‘adequate’,  rather  than  ‘excellent’,  was  documenting  juvenile  coral  density.  Documenting  the 
abundance  and  fate  of  juvenile  corals  is  a  key  monitoring  need,  as  the  ability  of  reef  communities 
to  persist  over  time  and  recover  from  disturbances  is  dependent  on  the  continuous  influx  of  coral 
recruits.  Image-based  approaches  in  general  are,  however,  not  as  good  at  documenting  recruits  as 
divers  because  recruits  and  juvenile  corals  are  very  small  (order  of  mm  to  cm)  and  can  be  easily 
obscured  from  view  by  larger  corals,  or  occur  on  bottom  surfaces  not  imaged  with  downward 
looking  cameras.  Nevertheless,  identification  and  monitoring  of  larger  juvenile  corals  (>  10cm) 
can  be  accomplished  using  2D  mosaics  and  other  image-based  approaches  (Lirman  et  al.  2007). 
Identification  of  juveniles  as  small  as  5  cm  is  better  on  second-generation  than  first-generation 
mosaics,  but  in  neither  case  is  as  good  as  diver  identification.  We  cannot  envision  development 
of  a  technology  in  the  near  future  that  could  replace  divers  for  monitoring  very  small  or  cryptic 
recruits.  Mosaics  are,  however,  useful  for  monitoring  juveniles  on  the  order  of  5-10  cm,  which 
could  be  adequate  for  many  coral  monitoring  needs. 

SI  1333  also  assessed  whether  landscape  mosaics  were  useful  in  overcoming  limitations  of  a 
diver-based  approach  to  coral  reef  assessment.  Six  metrics  were  evaluated  (required  experience; 
data  collection  time;  archive  potential;  depth  limitations;  mapping  large  scale  features;  and 
spatial  resolution;  Table  A14).  As  indicated  on  Table  A14,  even  the  initial  mosaics  excelled  in 
most  categories.  Advantages  of  mosaics  compared  to  the  diver  based  approach  and  other  imaging 
techniques  (i.e.  photo  quadrats  and  ID  strip  mosaics)  are  summarized  below. 

Field  personnel  lacking  the  scientific  expertise  required  for  diver-based  measurements  can 
conduct  Mosaic  surveys.  Imagery  can  be  acquired  using  regular  divers  or  with  unmanned 
platforms,  such  as  a  remotely  operated  vehicle  (ROY).  This  may  be  an  advantage  at  DoD 
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facilities,  where  trained  ROV  operators  may  be  more  available  than  divers  trained  as  coral  reef 
biologists! 

Data  acquisition  with  image-based  approaches  to  reef  monitoring  is  rapid  relative  to  data 
acquisition  by  trained  divers.  Relative  speeds  of  the  photo-quadrat,  strip  mosaics,  and  full  2-D 
mosaics  are  difficult  to  compare  because  times  will  be  dependent  on  survey  sampling  design.  As 
an  indication  of  speed,  however,  it  takes  about  30  minutes  to  acquire  the  video  for  a  10  m  x  10  m 
2-D  mosaic.  Diver  surveys  of  a  similar  site  could  take  hours  to  days. 

Image-based  techniques  for  reef  monitoring  have  excellent  archive  potential,  providing  a 
capability  that  is  vastly  superior  to  that  offered  by  written  notes  made  by  divers.  Image-based 
techniques  excel  because  variables  can  be  extracted  at  any  time  by  different  people.  In  addition, 
mosaics  are  superior  tools  to  track  patterns  of  change  over  time.  Mosaics  collected  in  repeat 
surveys  can  be  easily  referenced  to  one  another  with  only  four  permanent  markers  and  large 
numbers  of  coral  colonies  can  be  easily  monitored  over  time  without  the  need  for  extensive 
tagging  (Lirman  et  al.  2007).  This  way,  individual  colonies  can  be  followed  to  document 
changes,  such  as  burial,  or  removal  from  plots  (Lirman  et  al.  2006).  In  contrast,  tracking  changes 
over  time  with  photo-quadrats  requires  that  several  permanent  markers  be  deployed  per  image, 
which  is  tedious  to  set  up  and  easily  disturbed.  Strip  mosaics  can  be  re-run  with  permanent 
markers,  but  it  is  not  possible  to  ensure  that  exactly  the  same  area  is  covered  every  time.  The 
field  of  view  is  very  narrow  with  strip  mosaiced  imagery  (~0.5  m),  and  consequently  even  small 
drifts  can  result  in  large  portions  of  the  strips  not  being  covered  in  a  repeat  survey. 

Mosaics  are  also  ideal  for  documenting  reefs  at  or  beyond  the  depth  limit  for  scientific  SCUBA 
diving.  These  deep  reefs  are  increasingly  being  recognized  as  essential  reef  habitats.  We 
processed  mosaics  from  the  "Sherwood  Forest"  in  the  Dry  Tortugas  National  Park,  Florida  at  a 
depth  of  30  m  and  the  Marine  Conservation  District  of  the  U.S.  Virgin  Islands  at  depths  up  to  45 
m  (Gleason  et  al.  2009).  Since  image  data  may  be  acquired  from  platforms  such  as  ROVs  or 
AUVs,  there  are  no  theoretical  depth  limits  to  data  collection,  thereby  adding  considerable 
flexibility  to  a  reef  monitoring  program. 

As  a  further  benefit,  mosaics  provide  a  landscape  view  of  coral  reefs  that  has  previously  been 
unobtainable.  In  this  sense,  mosaics  bridge  a  gap  between  macro-scale  coral  reef  mapping  from 
airborne  imagery  and  detailed  underwater  monitoring  by  trained  divers.  Strip  mosaics  over  coral 
reefs  cover  typically  <  10m2  (Jaap  et  al.,  2002).  In  contrast,  2-D  mosaics  can  cover  in  excess  of 
400m2  of  reef  habitat  at  mm  scale  pixel  size  (ground  resolution).  This  landscape  view  of  the  reef 
makes  new  measures  of  reef  health  practical,  such  as  documenting  spatial  relationships  of 
disease  patterns,  or  the  effects  of  hurricane  damage  (Gleason  et  al.  2007)  and  ship  groundings 
(Lirman  et  al.  in  press). 

Finally,  mosaics  have  a  spatial  accuracy  on  the  order  of  a  few  centimeters  (Lirman  et  al.  2007). 
This  high  level  of  performance,  which  greatly  improves  upon  the  accuracy  of  strip  mosaics, 
combined  with  the  large  footprint,  enables  accurate  measurements  of  large  features,  such  as 
grounding  scars,  as  well  as  small  features  such  as  the  size  of  individual  coral  colonies.  In  corals, 
key  life-history  processes  such  as  growth,  mortality,  and  reproduction  are  directly  related  with 
colony  size.  Moreover,  the  age-structure  of  coral  populations  is  increasingly  being  used  as  an 
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indicator  of  disturbance  levels  on  reef  communities  (Bak  and  Meesters  1999).  Lirman  et  al. 
(2007)  showed  that  coral  colony  sizes  measured  from  mosaics  were  as  accurate  as  those 
measured  by  divers.  Lirman  et  al.  (in  press)  showed  that  two  mosaics  of  a  grounding  scar  in 
Biscayne  National  Park,  FL,  could  be  accurately  georeferenced  and  could  be  used  to  measure  the 
size  of  the  damaged  area  with  no  significant  difference  from  diver-based  measurements. 

The  results  above  indicate  that  the  landscape  mosaics  developed  within  project  SI  1333  address 
the  limitations  of  diver-based  coral  reef  assessment,  while  simultaneously  retaining  most  of  the 
strengths  of  a  diver-based  approach.  These  strengths  were  further  demonstrated  through  practical 
applications  to  a  variety  of  coral  reef  monitoring  efforts  (Section  A2.4).  The  spatial  accuracy  and 
archive  potential  of  landscape  mosaics,  for  example,  allows  individual  coral  colonies  to  be 
monitored  without  extensive  tagging;  this  characteristic  was  exploited  to  monitor  damage  due  to 
hurricanes  (Gleason  et  al.  2007;  Section  A2.4.4)  and  mass  bleaching  events  (Section  A2.4.5).  In 
addition,  the  landscape  view  of  mosaics  enables  visualization  and  measurement  of  large  features; 
this  characteristic  was  exploited  to  map  a  scar  from  a  ship  grounding  (Lirman  et  al.  in  press; 
Section  A2.4.1)  and  to  measure  the  size  of  a  large  slumping  feature  (Gleason  et  al.  2007;  Section 
A2.4.4).  The  short  dive  times  and  utility  of  mosaics  for  monitoring  deep  sites  was  also  explored 
in  the  imaging  of  mesophotic  reefs  (Gleason  et  al.  2009;  Section  A2.4.3). 

The  practical  applications  of  the  landscape  mosaic  technology,  as  outlined  above,  allowed  us  to 
test  the  system,  and  also  served  as  an  outreach  tool  to  coral  reef  science  community  (Section 
A2.5).  Each  of  the  applications  was  conducted  in  partnership  with  scientists  from  different 
organizations.  Over  the  course  of  the  project  we  conducted  field  operations  with  personnel  from 
eight  agencies  (or  groups  within  agencies).  The  SERDP  Coral  Reef  Monitoring  and  Assessment 
Workshop  and  the  project  web  site  were  sources  for  further  interactions  with  other  agencies. 

Additional  examples  illustrate  the  near-term  future  implications  of  the  mosaic  technology. 
NAYFAC  Engineering  and  Support  Center  Scientific  Diving  Services  (SDS)  would  like  to 
transition  the  landscape  mosaicing  technology  to  an  operational  product  to  use  in  routine 
monitoring  of  coral  reefs  near  Naval  bases.  As  a  first  step  in  making  this  transition,  SDS,  along 
with  partners  from  SPAWAR,  the  University  of  Miami,  and  the  University  of  Girona,  Spain, 
have  submitted  a  proposal  to  the  Environmental  Security  Technology  Certification  Program 
(ESTCP)  to  evaluate  the  operational  costs  and  performance  of  the  mosaics.  In  addition,  NOAA 
Assessment  &  Restoration  Division  is  planning  to  evaluate  the  use  of  landscape  mosaics  for 
documenting  the  December  2009  grounding  of  a  920’  tanker  on  a  reef  off  the  south  coast  of 
Puerto  Rico.  Finally,  future  use  of  landscape  mosaics  may  extend  to  applications  beyond  coral 
reef  mapping  and  monitoring.  For  example,  NOAA’s  Ocean  Service  and  Office  of  National 
Marine  Sanctuaries  are  considering  the  use  of  mosaics  to  assist  mapping  and  monitoring  of 
discarded  military  munitions  off  Oahu  Hawaii. 

In  summary,  landscape  mosaics  developed  under  SI  1333  provide  scientists  and  resource 
managers  with  novel,  powerful  survey  tools.  Project  deliverables  include  software  to  generate 
mosaics,  tools  to  facilitate  viewing  the  mosaics  and  assist  in  the  extraction  of  ecological 
information,  and  user  manuals.  Landscape  mosaics  can  now  be  used  to  routinely  map  large 
underwater  areas  at  high  spatial  resolution  (on  the  order  of  400  m2  at  sub-mm  resolution), 
resulting  in  spatially  accurate,  landscape  views  of  the  bottom  that  were  previously  unobtainable. 
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These  landscape  mosaics  will  be  useful  for  DoD  reef  monitoring  requirements  and  open  doors 
for  new  applications  in  reef  mapping  and  change  detection. 


A4.  Action  Item 

In  your  final  Report  provide  an  operational  cost  analysis  between  the  ROV/mosaic/3D  technique 
and  the  current  industry  standard  using  divers  and  2D  data. 

Response: 

As  an  action  item  from  a  previous  IPR,  we  were  asked  to  provide  an  operational  cost  analysis  of 
the  landscape  mosaic  technique  developed  in  the  present  project  (SI  1333)  in  comparison  to 
current  industry  standard  reef-survey  methodologies  that  utilize  divers  to  conduct  assessments  of 
reef  condition. 

At  present  there  is  not  a  single  industry  standard  for  reef  surveys  -  the  method  for  any  reef 
monitoring  protocol  necessarily  depends  on  the  particular  questions  being  asked.  We  have 
chosen  to  compare  the  cost  of  conducting  reef  surveys  using  landscape  video  mosaics  with 
traditional  diver-based  methods,  such  as  those  that  were  used  to  document  reef  status  at  AUTEC 
in  the  70s  and  80s,  and  a  more  recent  rapid  reef  survey  methodology  (AGRRA),  used  currently  at 
AUTEC. 

A  direct  comparison  of  the  cost  of  each  survey  methodology  is  hindered  by  the  fact  that  while 
some  of  the  indicators  collected  are  common  to  all  three  methods  (see  Table  A15  for  a  list  of 
indicators),  there  are  unique,  method-specific  indicators  that  need  to  be  considered  in  the  cost 
comparison.  The  cost  of  each  survey  needs  to  be  weighed  according  to  the  diversity,  accuracy, 
and  precision  of  the  indicators  being  collected  as  well  as  the  potential  application  of  the  data 
collected.  In  this  report,  we  provide  a  list  of  the  indicators  collected  by  each  method  as  well  as  a 
description  of  the  applications  and  advantages  and  disadvantages  of  each  methodology  that  help 
qualify  the  cost  estimates  shown  in  Table  A16. 

Diver-Based  Mapping  Methodology 

The  goal  of  the  diver-based  mapping  methodology,  used  by  NAVO  at  AUTEC  during  the  70s 
and  80s  to  monitor  status  and  trends  of  coral  reefs,  is  to  map  the  abundance  and  distribution  of 
corals  and  other  organisms  within  marked,  permanent  plots.  These  plots  are  re-visited 
periodically  to  evaluate  changes  over  time.  The  emphasis  of  this  methodology  is  on  within-site 
precision  and  accuracy  as  well  as  repeatability.  The  final  product  of  this  methodology  is  a  hand- 
drawn  map  of  the  reef  bottom.  The  location  and  size  of  individual  coral  colonies  or  other  features 
is  determined  by  divers  in  the  field  using  flexible  underwater  tapes.  The  metrics  collected  during 
these  surveys  include  coral  abundance,  diversity,  and  colony  sizes.  Other  organisms  such  as 
sponges  and  soft  corals  are  also  included  in  the  maps.  Once  developed,  these  maps  can  be  used 
to  measure  the  benthic  coverage  of  the  main  reef  organisms. 

The  positive  aspects  of  the  diver-based  mapping  methodology  are:  (1)  the  maps  collected  using 
this  methodology  provide  a  landscape  sketch  of  the  reef  bottom  that  can  be  archived  for 
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reference  and  change  detection  over  time;  and  (2)  the  identification  of  individual  coral  colonies 
within  plots  provides  a  high  accuracy  and  precision  to  detect  changes  over  time. 


The  negative  aspects  of  this  methodology  are:  (1)  the  time  required  to  complete  an  assessment  of 
a  typical  permanent  site  (2  days  for  a  plot  roughly  20  m  x  20  m)  limits  the  number  of  sites  that 
can  be  practically  surveyed  within  a  region  during  a  monitoring  mission,  (2)  the  divers 
conducting  the  surveys  need  to  be  trained  in  species  identification;  (3)  the  spatial  accuracy  of  the 
maps  is  limited  because  they  are  hand-drawn  by  divers  in  the  field;  (4)  the  field  maps  need  to  be 
transferred  into  digital  format  for  analysis  using  digitizing  software;  and  (5)  the  lack  of 
replication  due  to  the  long  time  required  to  survey  each  site  influences  the  representative-ness  of 
the  observed  patterns,  which  can  not  be  easily  extrapolated  to  the  regional  level. 


SURVEY  METHODS 

NAVO 

AGRRA 

MOSAICS 

Indicators 

Coral  Cover 

Y 

Y 

Y 

Coral  Diversity 

Y 

Y 

Y 

Coral  Colony  Diameter 

Y 

Y 

Y 

Recent  Mortality 

N 

Y 

Y 

Old  Mortality 

N 

Y 

Y 

Juvenile  Coral  Density 

Y 

Y 

Y 

Macroalgal  Cover 

N 

Y 

Y 

Sponge  Cover 

Y 

N 

Y 

Soft  Coral  Cover 

Y 

N 

Y 

Topography 

N 

Y 

N 

Macroalgal  Canopy  Height 

N 

Y 

N 

Spatial  Patterns 

Y 

N 

Y 

Disease  Prevalence 

N 

Y 

Y 

Bleaching  Prevalence 

N 

Y 

Y 

Uchin  abundance 

N 

Y 

N 

Data  Collection 

Field 

2  days 

2  hrs 

30-40  min 

Lab 

2  days 

4  hrs 

2  days 

Number  of  Divers 

4-5 

4 

2 

Training  Level 

Field  Personnel 

HIGH 

HIGH 

LOW 

Lab  Personnel 

LOW 

LOW 

HIGH 

Other 

Spatial  Accuracy 

LOW 

N/A 

HIGH 

Permanent  Record 

Y 

N 

Y 

Landscape  Image 

hand-drawnmap 

N 

digital  image 

Products 

Data 

Data 

Data 

Digital,  spatially 
accurate  high- 

Maps 

resolution 
landscape 
_ Lmaofis _ 

Table  A15.  Methods  comparison.  Overview  of  the  characteristics  of  each  survey  method,  including  the  reef 
condition  indicators  extracted,  the  effort  required  for  data  collection,  the  training  required  for  personnel,  and 
other  survey  characteristics. 


AGRRA  (Atlantic  and  Gulf  Rapid  Reef  Assessment) 


129 


RC-1333  Final  Report  to  SERDP  2010 


R.P.Reid 


The  main  goal  of  the  AGRRA  methodology  is  to  document  large-scale  patterns  of  reef  condition 
using  rapid  in-water  surveys  to  collect  indicators  of  coral  reef  condition.  The  information 
gathered  during  AGRRA  surveys,  which  is  focused  on  stony  corals,  includes  coral  diversity, 
abundance,  condition  (recent  and  old  partial  mortality),  and  colony  sizes.  Other  metrics  include 
percent  cover  of  corals,  macroalgae,  and  other  organisms,  as  well  as  macroalgal  cover  canopy 
height  and  reef  topography.  The  prevalence  of  coral  diseases,  bleaching,  and  the  abundance  of 
sea  urchins  ( Diadema )  are  also  noted.  Finally,  the  potential  causes  of  coral  mortality  (e.g., 
sponge  overgrowth,  macroalgal  overgrowth,  sediments)  are  documented. 

The  positive  aspects  of  this  methodology  are:  (1)  the  bottom  time  at  a  given  site  is  commonly 
short  (<  2  hrs)  allowing  for  multiple  sites  to  be  surveyed  in  one  day;  (2)  to  date,  >  700  sites  have 
been  surveyed  with  this  methodology,  providing  a  regional  baseline  against  which  new  surveys 
can  be  compared;  and  (3)  during  AGRRA  surveys,  all  of  the  reef  condition  indicators  are 
collected  in  situ.  Data  transcription  and  analyses  are  conducted  at  a  later  time  in  the  lab,  but  no 
lab  time  is  required  for  extracting  indicators  (e.g.  coral  cover)  from  raw  data  (e.g.  video). 

The  negative  aspects  of  this  methodology  are:  (1)  within-site  accuracy  and  precision  are  de- 
emphasized  in  lieu  of  spatial  coverage  and  number  of  sites  sampled;  (2)  the  ability  of  this 
methodology  to  detect  changes  over  time  has  not  been  tested,  as  the  number  of  repeat  surveys  of 
the  same  areas  has  been  limited  to  date;  (3)  corals  <  10  or  20  cm  in  diameter  are  often  excluded 
to  expedite  the  surveys;  and  (4)  the  divers  conducting  AGRRA  surveys  need  to  be  trained  in 
species  identification  and  go  through  a  rigorous  calibration  training  exercise. 

Landscape  Video  Mosaics 

Landscape  video  mosaics  are  used  to  obtain  a  high  resolution  (mm-scale  pixels)  composite 
image  of  a  large  area  (hundreds  of  mA2)  of  the  seabed  that  allows  for  the  direct  extraction  of 
coral  indicators  such  as  coral  cover  and  colony  sizes.  This  methodology  provides  a  bridge 
between  rapid  and  comprehensive  reef  survey  methodologies  by  requiring  only  a  short  time  of 
data  collection  in  the  field  and  shifting  the  assessment  load  from  the  field  to  the  lab.  Moreover, 
video  mosaics  can  be  analyzed  to  extract  the  same  general  suite  of  indicators  of  reef  condition 
that  are  collected  with  the  AGRRA  and  diver-based  mapping  methods,  but  they  also  provide  a 
unique  set  of  metrics  and  products  that  can’t  be  collected  using  the  former  methods  (Table  A15). 
The  final  product  of  a  video  mosaic  survey  is  a  spatially  accurate  landscape  digital  image  of  the 
reef  bottom  that  can  be  analyzed  to  extract  indicators  of  reef  condition. 

The  positive  aspects  of  this  methodology  are:  (1)  digital  video  mosaics  offer  all  of  the 
advantages  of  diver-drawn  maps  such  as  high  precision  and  accuracy  to  detect  within-site 
changes  over  time;  (2)  mosaics  provide  the  added  capability  to  extract  accurate  size  and  distance 
measurements  directly  from  the  images,  a  unique  tool  to  document  and  assess  the  impacts  of 
disturbances  such  a  ship  groundings  or  anchor  damage  that  are  hard  to  assess  in  the  field;  (3)  reef 
plots  can  be  surveyed  very  quickly  (<  40  min  by  a  single  camera  operator),  thus  allowing  for 
multiple  sites  to  be  surveyed  in  a  given  mission;  (4)  camera  operators  do  not  need  any  prior 
ecological  training;  (5)  this  methodology  shifts  the  analysis  time  form  the  field  to  the  lab  where 
operations  are  generally  cheaper  and  less  time-constrained;  and  (6)  the  end  product  provides  a 
powerful  visual  tool  that  can  be  easily  archived  for  future  reference. 
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The  negative  aspects  of  this  methodology  are:  (1)  the  construction  of  mosaics  can  take  up  to  48 
hrs  in  the  lab  and  requires  qualified  technicians;  (2)  because  only  the  top  surface  of  the  reef  is 
captured  in  the  video,  cryptic  habitats  can’t  be  surveyed;  and  (3)  video  surveys  can’t  be 
conducted  in  conditions  of  limited  visibility. 

Cost  Comparison  Among  Survey  Methodologies 

The  relative  costs  of  each  survey  method  were  estimated  from  four  components  of  the  survey 
process:  boat  time,  diver  time,  data  processing  time,  and  analysis  time.  Boat  cost  was  computed 
as  the  price  of  boat  rental  for  one  day  (assumed  to  be  $1000)  divided  by  the  number  of  sites  that 
could  be  surveyed  in  one  day.  The  number  of  sites  possible  in  one  day  was  based  on  field 
experience  and  assumed  favorable  conditions  (e.g.  short  distances  between  sites,  no  problems 
anchoring,  good  weather  and  so  forth).  Diver  cost  was  computed  as  the  number  of  divers 
required  times  the  cost  per  day  per  diver  (assumed  to  be  $500)  divided  by  the  number  of  sites  per 
day.  Data  processing  time  includes  data  transcription  from  field  sheets  to  electronic  format  (for 
NAVO  and  AGRRA  methods)  and  mosaic  creation  (Mosaic  method  only).  For  the  mosaics, 
computer  time  required  to  create  the  mosaic  is  not  included,  only  the  time  required  for  interactive 
input  from  the  technician.  Extracting  indicators  from  the  NAVO  and  AGRRA  data  is  rapid,  once 
the  data  are  in  digital  form.  Extracting  indicators  from  the  mosaics  involves  point  counting  so 
takes  longer  than  the  other  methods.  Technician  costs  for  data  processing  and  indicator 
extraction  were  estimated  at  $500/day.  Costs  for  consumable  supplies  such  as  waterproof  paper, 
video  tapes,  PVC  quadrats  and  so  on  are  small  when  calculated  on  a  per-site  basis  ($10  or  so),  so 
were  not  included  in  these  calculations. 

The  depth  of  survey  sites  is  an  important  variable  to  consider  for  planning  diving  operations. 
Depth  will  dictate  the  required  surface  intervals  between  dives  and  therefore  impact  the  number 
of  sites  that  can  be  visited  in  a  single  day.  In  order  to  capture  this  element  of  cost  we  have  made 
estimates  of  the  number  of  sites  possible  per  day  in  three  depth  ranges:  0-30  feet,  30-50  feet,  and 
50-90  feet.  The  cutoff  at  30'  was  chosen  because  shallower  than  30'  bottom  time  is  not  a  limiting 
factor.  The  cutoff  at  50'  was  chosen  because  virtually  no  AGRRA  surveys  have  been  conducted 
below  that  depth.  The  cutoff  at  90'  was  chosen  as  the  maximum  depth  at  which  a  diver  on  air 
would  have  enough  time  to  realistically  complete  acquisition  of  a  lOmx  10m  mosaic  in  one 
dive. 

As  shown  on  Table  A16,  the  most  expensive  method  is  diver-based  maps,  at  an  estimated  cost  of 
$7700  per  site.  Below  30  feet  it  is  not  practical  to  complete  diver-based  maps.  In  shallow  water, 
the  AGGRA  method  is  the  least  expensive,  at  an  estimated  cost  of  $1 100  per  site.  The  mosaicing 
method  is  slightly  more  expensive  than  AGGRA,  at  a  cost  of  approximately  $1350  per  site  in 
shallow  water.  In  the  middle  depth  range,  30-50',  the  mosaics  become  slightly  less  expensive 
than  AGRRA.  Below  50  feet,  the  mosaics  are  the  most  cost  effective  method. 
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SURVEY  METHODS  I 

Survey  Components 

NAVO 

AGRRA 

MOSAICS 

Boat 

N  sites  per  day  {<  30') 

05 

4 

6 

N  sites  per  day  (30'  -  50') 

N/A 

2 

6 

N  sites  per  day  (50‘  -  90') 

N/A 

N/A 

4 

Cost  per  site  (<  30') 

2000 

250 

167 

Cost  per  site  (30 1  -  50’) 

N/A 

500 

167 

Cost  per  site  (5Q1  -  90s) 

N/A 

N/A 

250 

Divers 

N  divers  {<  30) 

4 

4 

2 

N  divers  (30’  -  50') 

N/A 

4 

4 

N  divers  (50'  -  90') 

N/A 

N/A 

4 

Cost  per  she  (<  30’) 

4000 

500 

167 

Cost  per  site  (301  -  50') 

N/A 

1000 

333 

Cost  per  site  (5G1  -  90') 

N/A 

N/A 

500 

Data  Processing 

Technician-Days  of  processing  per  site 

4 

0.5 

1 

Cost 

2000 

250 

500 

Extracting  Indicators 

T echnician-Days  of  analysis  per  site 

02 

02 

t 

Cost 

100 

100 

500 

Total  per  Site  (<  30') 

3100 

1100 

1333 

Total  per  Site  (301  -  50‘) 

N/A 

1850 

1500 

Total  per  Site  (50T  -  90p) 

N/A 

N/A 

1750 

Table  A16:  Estimates  costs  per  site  of  three  reef  survey  methodologies.  The  costs  assumed  were:  Boat  = 
$1000/day,  Diver  =  $500/day,  Data  processing  and  indicator  extraction  =  $500/day. 


Conclusion 


The  products  of  the  mosaicing  method  are  comparable  to  the  diver-based  maps,  but  are  much 
cheaper  to  produce.  AGRRA  surveys  are  less  expensive  than  mosaics  in  shallow  water  but 
become  more  expensive  than  mosaics  in  deeper  water.  Additionally,  mosaics  overcome  some 
limitations  of  the  AGGRA  method,  providing  within-site  accuracy,  ability  to  detect  changes  over 
time  through  repeat  surveys,  detection  of  small  corals  and  no  need  for  scientifically  trained 
divers. 
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Section  B:  Single  Object  3D  Reconstruction 

To  assist  the  classification  of  landscape  mosaics,  a  secondary  goal  of  project  SI  1333  was  to 
investigate  3D  reconstruction  of  single  objects  within  the  landscape  mosaics.  3D  metrics  such  as 
height  and  rugosity  can  provide  users  with  information  on  the  topographical  complexity  created 
by  corals,  which  can  be  used  to  identify  key  habitat  features  for  a  variety  of  organisms. 
Additionally,  the  3D  structure  of  coral  colonies  is  important  in  many  physiological  processes 
such  as  coral  calcification,  growth,  mortality,  and  fecundity.  Despite  importance  of  topography, 
3D  coral  colony  structure  is  not  commonly  incorporated  in  monitoring  programs  because  it  is 
extremely  difficult  to  measure. 

Research  conducted  within  SI  1333  developed  a  3D  reconstruction  tool  that  allows  a  user  to 
visualize  and  measure  topographic  structure  and  heights  of  single  objects,  such  as  coral  colonies, 
from  landscape  mosaics  created  from  stereo  imagery.  With  a  user-friendly  interface,  the  software 
relies  on  the  camera  trajectory  and  indexed  images  used  for  mosaic  construction.  Starting  with 
the  identification  of  an  object  of  interest  in  the  mosaic,  the  user  follows  a  number  of  steps  that 
lead  to  the  construction  of  a  3D  map  of  the  coral  colony  or  specific  area  of  interest.  The  results 
obtained  using  the  3D  reconstruction  software  compare  favorably  with  diver  measurements. 
Overall  our  research  has  demonstrated  that  the  3D  system  is  a  useful  tool  that  can  automatically 
determine  object  sizes  and  small  scale  rugosity  measurements  from  stereo  views. 

Bl.  Materials  and  Methods 

In  order  to  make  3D  measurements,  a  3D  map  of  an  object  or  region  of  interest  must  be 
constructed.  As  part  of  SI  1333,  we  investigated  two  approaches:  1)  an  optical  flow-based 
method  applied  to  monocular  video  sequence  for  the  dense  estimation  of  a  3D  map,  and  2)  a 
feature-based  technique  allowing  3D  reconstruction  of  single  objects.  We  determined  that  the 
second  approach  was  more  suitable  for  developing  a  tool  for  non-expert  use  in  coral  reef 
applications. 

Simple  geometric  triangulation  provides  the  foundation  for  feature  based  3D  reconstruction  of 
stereo  video  imagery.  Consider  a  pair  of  overlapping  images  taken  from  slightly  different  points. 
If  the  baseline  between  the  focal  points  of  the  two  images  and  the  transformation  from  pixel  row 
and  column  to  angle  from  the  camera  optical  axis  are  known,  then  the  distances  from  each 
camera  to  a  single  point  visible  in  both  images  can  be  computed  geometrically. 

Given  stereo  images  acquired  in  a  down-looking  configuration  from  above  an  object,  the 
differences  between  the  z-coordinates  of  a  point  near  the  top  of  the  object  and  one  near  the  base 
of  the  object  give  the  approximate  height  of  the  object.  When  only  a  few  points  are  matched 
between  a  pair  of  images,  the  result  is  a  "sparse  depth  map".  On  the  other  hand,  when  nearly 
every  pixel  in  one  image  can  be  matched  with  a  pixel  in  the  other  image  the  result  is  a  "dense 
depth  map",  which  is  essentially  an  elevation  model  or  terrain  model  of  the  surface  in  the 
images.  Even  a  dense  depth  map,  however,  is  not  a  full  three  dimensional  reconstruction  like 
those  computed  by  Bythell  et  al.  (2001)  and  Cocito  et  al.  (2003)  because  in  a  single  overhead 
stereo  pair,  there  will  generally  be  occluded  areas  that  are  not  imaged  and  can  therefore  not  be 
triangulated  (reconstructed). 


133 


RC-1333  Final  Report  to  SERDP  2010 


R.P.Reid 


Our  methodology  for  feature-based  3D  reconstruction  of  single  objects  is  as  follows:  First, 
construct  and  calibrate  an  underwater  stereo  camera.  Second,  acquire  overhead  stereo  images  of 
the  survey  area.  Third,  process  the  stereo  pairs  to  form  dense  depth  maps  and  then  select  points 
near  the  top  and  bottom  of  each  object  of  interest  to  get  the  heights  of  the  objects.  Further  details 
are  given  below. 

The  stereo  imaging  system  used  for  this  work  was  custom-built  using  commercially  available 
components  as  much  as  possible.  The  imaging  device  was  a  Videre  Designs  MDCS2  variable 
baseline  digital  stereo  head  set  to  an  approximate  baseline  of  20  cm.  A  VIA  EPIA  Mini  ITX 
computer  running  RedHat  Linux  9.0  performed  all  control  functions.  The  stereo  head  was  driven 
over  an  IEEE-1394  connection  by  the  Small  Vision  Systems  (SVS)  version  4.1  software  library 
that  was  provided  with  the  cameras  to  perform  data  acquisition.  The  standard  interface  to  this 
library  had  to  be  modified  to  accept  control  from  a  touch  pad  rather  than  the  default  mouse- 
driven  interface.  During  acquisition,  images  were  written  in  raw  format  to  an  80  GB  hard  disk. 
Image  acquisition  averaged  about  5  frames  per  second.  When  downloaded  at  the  end  of  the  day 
the  images  were  converted  to  color  at  1280x960  resolution. 

The  stereo  video  system  was  calibrated  using  the  Camera  Calibration  Toolbox  for  Matlab 
(Bouguet  2002).  The  calibration  routine  solved  for  the  intrinsic  parameters  for  each  camera 
(focal  length,  lens  distortion,  pixel  aspect  ratio,  center  pixel  coordinates)  as  well  as  the  extrinsic 
parameters  (rotation  and  translation)  relating  the  coordinate  systems  of  the  two  cameras. 

Image  processing  consists  of  six  steps  for  each  stereo  pair:  image  selection,  image  pre¬ 
processing,  feature  matching,  match  propagation,  interpolation,  and  height  extraction.  The  first 
three  steps  create  a  "sparse  depth  map"  as  defined  above.  The  fourth  and  fifth  steps  fill  in  the 
areas  between  the  matched  features  to  create  a  "dense  depth  map". 

Image  selection  involves  reviewing  the  captured  images  and  extracting  only  the  stereo  pairs  to  be 
processed:  namely,  those  containing  the  objects  marked  by  the  divers.  We  have  built  a  graphical 
interface  for  image  selection,  within  which  the  images  are  first  stitched  together  into  a  mosaic 
using  the  methods  described  in  Lirman  et  al.  (2007)  and  then  a  user  clicks  on  objects  of  interest 
(numbered  tiles  in  this  case)  to  bring  up  all  stereo  pairs  that  overlapped  that  point. 

The  second  step  is  image  pre-processing,  in  which  the  camera  calibration  parameters  are  applied 
to  each  stereo  pair  to  a)  remove  lens  distortion  effects,  and  b)  rectify  the  images  such  that 
matching  features  would  appear  on  the  same  row  in  each  image  (Bouguet  2002). 

The  third  step  is  feature  matching,  in  which  specific  features  visible  in  both  images  are  identified 
as  belonging  to  the  same  physical  point  on  the  seabed.  The  Shift-Invariant  Feature  Transform 
(SIFT)  algorithm  is  used  to  automatically  match  features  in  each  pair  of  images  (Lowe  2004). 
The  SIFT-generated  matches  are  reviewed  to  discard  any  incorrect  matches  and  flag  any  areas  of 
the  image  where  SIFT  failed  to  find  matches. 

Match  propagation,  the  fourth  step,  uses  each  of  the  SIFT-generated  matches  as  a  seed  point  to 
look  for  additional  matches  within  a  small  window  around  that  point.  The  idea  is  to  search  for 
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additional  matching  points  near  points  known  to  be  good  matches.  The  algorithm  implemented 
for  match  propagation  is  guided  local  disparity  computation  (Chen  and  Medioni  1999). 
Theoretically,  if  the  distances  to  the  cameras  vary  slowly,  and  if  there  is  adequate  texture  across 
the  image,  then  match  propagation  could  fill  in  matches  across  the  entire  image  from  a  single 
seed  point.  In  practice,  depth  map  discontinuities  due  to  occlusions  or  illumination  artifacts 
violate  these  constraints  and  match  propagation  can  fill  in  only  a  local  region  around  each  seed 
point.  Therefore,  having  many  SIFT-generated  seed  points  to  start  with  helps  this  process 
succeed. 

The  fifth  step  interpolates  between  matched  points  to  fill  in  gaps  following  match  propagation. 
The  result  after  the  fifth  step  is  a  dense  depth  map  computed  for  the  area  of  overlap  between  the 
two  images.  The  final  step  is  to  identify  the  top  and  bottom  of  the  object(s)  of  interest  and 
compute  the  depth  difference  between  those  points.  A  graphical  interface  was  designed  to 
facilitate  this  process.  The  analyst  clicks  with  a  mouse  on  the  pixels  at  the  top  of  the  object  of 
interest  and  on  the  benthos  surrounding  the  object  to  calculate  and  record  the  vertical  height  of 
the  object. 


B2.  Results  and  Discussion 

As  a  culmination  of  our  work  in  3D  mosaicing  we  developed  a  “point  and  click”  system  that 
allows  users  to  select  features  of  interest  from  a  2D  video  mosaic  (created  from  stereo  imagery) 
and  create  a  small-scale  3D  interpolation  of  that  object.  The  steps  involved  in  using  the  “ point 
and  click”  system  are  described  in  methods. 

Sample  results  are  given  in  Figures  B1-B7.  In  Figure  Bl,  elevation  has  been  encoded  by  color 
(red  highlights  low  elevation,  and  purple  is  the  highest  elevation);  the  elevation  display  is 
generated  using  the  GeoZui  system  (University  of  New  Hampshire  website).  The  representations 
allow  users  to  click  on  map  features  and  examine  their  elevations,  or  click  on  pairs  of  points  on 
the  map  to  determine  the  elevation  difference.  This  feature  allows  us  to  determine  the  3D  height 
of  corals  or  other  benthic  objects  directly  from  mosaic  products. 

Figure  B2  depicts  an  example  of  various  steps  of  the  point-click  system.  In  (b),  we  have  shown 
both  features  detected  and  matched  by  the  algorithm,  as  well  as  those  added  manually  over  the 
coral  (within  rectangular  region).  The  disparity  map  in  (c)  has  been  shown  by  gray-level  coding, 
with  the  gray  bar  code  given  to  the  right  of  the  map.  The  horizontal  and  vertical  distances 
between  various  points  in  (d)  are  determined  by  clicking  on  pairs  of  points  on  the  image.  Using 
this  technology  we  can  select  two  points  within  an  image  and  view  the  regional  rugosity  between 
the  points  of  interest.  Figure  B3  shows  the  selection  of  various  lines  of  interest  (for  examining 
regional  rugosity)  in  four  different  reef  scenes.  Figure  B4  shows  the  results  of  the  rugosity 
computations  of  these  scenes  from  the  3D  map. 


135 


RC-1333  Final  Report  to  SERDP  2010 


R.P.Reid 


(a)  (b)  (c) 


Figure  Bl.  A  sample  stereo  pair,  shown  in  (a)  and  (b),  are  processed  by  our  method  to  generate  the  3D  map 
in  (c).  Here,  colors  encode  height,  with  red  the  low  points  and  purple  the  highest  elevation. 
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a) 


b) 


c)  d) 

Figure  B2.  Sample  stereo  pair  with  matching  features  (a).  Features  over  the  coral  reef,  within  yellow 
rectangle  have  been  added  manually  (b).  Computed  disparity  map  in  (c)  allows  the  reconstruction  of  3D 
map,  and  measurement  of  D  distances  (d). 
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Figure  B3.  Sample  left  view  of  a  stereo  pair,  showing  the  corals  that  have  been  processed  for  3D 
reconstruction. 
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Figure  B4.  Scan  lines  from  the  3D  reconstruction  of  corals  in  figure  B3. 

Accuracy  of  the  3D  system  has  been  assessed  by  comparing  measurements  with  those  made  by 
two  divers,  as  well  as  known  sizes  (14.5”  and  19”)  of  a  number  of  flower  pots  placed  on  the  sea 
floor.  The  height  has  been  determined  from  the  top  of  the  object  to  a  nearby  point  on  the  sea 
floor.  For  the  corals,  these  two  locations  were  marked  by  a  2”  round  metallic  washer  and  a 
4.3  ”x  4.3”  tile,  respectively.  Height  estimates  using  the  point  and  click  system  for  coral  and 
reef  rubble  are  compared  to  two  independent  diver  measurements  in  Figure  B5.  Comparison  of 
the  point  and  click  system  to  objects  of  known  size  are  shown  in  Figure  B6. 
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Figure  B6.  Control  points  for  3D  point  and  click  calibration.  Flower  pots  of  known  dimension  were  placed 
on  the  seafloor  and  measured  in  the  same  way  as  coral  colonies  or  rubble.  Height  estimates  from  the  point 
and  click  system  (3D  column)  are  compared  to  that  of  laboratory  measurements  of  the  pots  (G-truth  column). 

In  all  but  three  cases,  the  results  show  that  the  estimates  from  the  3D  system  are  in  close 
agreement  with  the  two  diver  measurements,  as  well  as  the  pot  sizes.  In  one  case,  tile  5 1  in  the 
2nd  data  set,  one  diver  measurement  agrees  well  with  the  estimate  from  the  3D  system  (97. 1  [cm] 
-vs-  99  [cm]),  but  varies  by  roughly  10%  from  the  2nd  diver  measurement  (108  [cm]).  In  another 
case,  tile  20  of  2nd  data  set,  the  discrepancy  is  roughly  25%  (25.8  [cm]  -vs-  average  diver 
measurement  of  33  [cm]).  Only  in  one  case,  tile  44  in  data  set  1,  the  3D  system  and  diver 
measurements  disagree  significantly  (58.5  [cm]  -vs-  average  diver  measurement  of  39  [cm]). 
Given  that  3  independent  estimates  by  the  3D  system  have  produced  consistent  results  (see  next 
paragraph)  and  the  two  diver  measurements  are  within  2  [cm],  such  a  large  discrepancy  can  only 
be  attributed  to  some  unknown  factor  that  we  have  not  been  able  to  identify  (such  as  recording 
error). 

The  consistency  and  (or)  repeatability  of  the  3D  system  was  investigated  by  determining  the 
height  for  selected  corals  from  different  stereo  pairs.  More  precisely,  we  have  applied  our 
method  to  3  different  stereo  pairs  of  the  same  object  recorded  from  nearby  positions  of  the  stereo 
platform.  Figure  B7  shows  these  results.  The  standard  deviation  of  the  three  measurements  for 
each  object  is  equal  or  less  than  2.5%  of  the  average  estimate  in  all  but  one  case,  where  it  is 
5.8%.  From  these  results,  one  can  conclude  that  the  3D  system  is  a  useful  tool  to  determine  the 
missing  height  dimension  automatically  from  a  pair  of  stereo  views. 
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3  Indep.  3D  Measurements 


2  Indep.  Diver  Measurements 


Figure  B7.  Independent  measurements  of  the  same  coral  colony  or  reef  rubble  by  the  3D  point  and  click 
system  (3  different  stereo  pairs)  and  2  diver  measurements. 


B3.  Conclusions  and  Implications  for  Future  Research/Implementation 

Our  overall  mosaicing  research  has  targeted  the  development  of  image-based  technologies,  to 
increase  the  speed  and  repeatability  with  which  reef  plots  can  be  mapped  and  inventoried. 
Because  of  habitat  complexity,  monitoring  tools  that  document  the  3D  topography  can  provide 
valuable  information  on  reef  structure  and  function  which  are  not  commonly  obtained  by 
standard  monitoring  methodologies  for  measuring  coral  cover  as  the  proportion  of  the  bottom 
occupied  by  corals  in  a  planar  view.  More  precisely,  utilizing  our  tool  for  the  3D  size 
measurements,  the  user  can  more  readily  acquire  the  minimum  technical  knowledge  for 
assessing  the  performance  of  the  underlying  methods  comprising  the  entire  system. 

The  3D  tool,  which  has  a  user-friendly  GUI,  is  applied  to  a  2D  mosaic  of  a  reef  area.  It  relies  on 
the  camera  trajectory  and  the  indexed  images  that  are  utilized  in  the  construction  of  the  mosaic. 
Starting  with  the  identification  of  an  object  of  interest  in  the  mosaic,  the  user  can  follow  a 
number  of  simple  consecutive  steps  leading  to  the  construction  of  the  final  3D  map.  These  steps 
allow  the  user  to  examine  each  intermediate  result,  which  as  a  product  can  be  readily  assessed  by 
a  non-expert.  For  example,  choosing  between  two  different  methods  for  establishing  the 
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correspondences  in  the  two  stereo  images,  he/she  can  verify  if  the  stereo  matches  have  been 
identified  correctly.  The  user  can  readily  remove  outliers  if  they  exist.  Alternatively,  in  regions 
where  our  system  fails  to  identify  correspondences,  the  user  can  manually  add  matched  features. 

The  results  obtained  from  our  3D  system  have  been  compared  to  the  heights  measured  by  two 
divers  using  flexible  tapes,  as  well  as  the  flower  pots  of  know  sizes  laid  on  the  sea  floor.  The  3D 
estimates  are  in  good  agreement  with  independent  diver  measurements  and  the  true  pot  sizes  (see 
Appendix  5).  Furthermore,  repeatability  of  the  estimates  from  the  3D  system  has  been 
demonstrated  by  applying  our  method  to  three  different  stereo  pairs  for  each  of  several  objects 
(Appendix  5).  Our  investigation  has  demonstrated  that  the  3D  system  is  a  useful  tool  to 
determine  object  sizes  automatically  from  stereo  views. 
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Section  C:  Multispectral  Imaging 

A  method  for  automated  classification  of  reef  benthos  would  improve  coral  reef  monitoring  by 
reducing  the  cost  of  data  analysis.  Spectral  classification  of  standard  (three-band)  color 
underwater  imagery,  however,  does  not  work  well  for  distinguishing  major  bottom  types.  Recent 
publications  of  hyperspectral  reflectance  of  corals,  algae,  and  sediment,  on  the  other  hand, 
suggest  that  careful  choice  of  narrow  (~10  nm)  spectral  bands  might  improve  classification 
accuracy  relative  to  the  three  wide  bands  available  on  commercial  cameras.  We  built  an 
underwater  multispectral  camera  to  test  whether  narrow  spectral  bands  were  actually  superior  to 
standard  RGB  cameras  for  automated  classification  of  underwater  images.  A  filter  wheel  was 
used  to  acquire  imagery  in  six  10  nm  spectral  bands,  which  were  chosen  from  suggestions  in  the 
literature. 

Results  indicate  that  the  algorithms  for  classifying  underwater  imagery  suggested  in  the  literature 
require  careful  compensation  for  variable  illumination  and  water  column  attenuation  for  even 
marginal  success.  On  the  other  hand,  research  in  project  SI  1333  suggests  that  a  new  algorithm, 
based  on  the  normalized  difference  ratio  of  images  at  568  nm  and  546  nm  can  reliably  segment 
photosynthetic  organisms  (corals  and  algae)  from  non-photosynthetic  background.  Moreover, 
when  this  new  algorithm  is  combined  with  very  simple  texture  segmentation,  the  general  cover 
classes  of  coral  and  algae  can  be  discriminated  from  the  image  background  with  accuracies  on 
the  order  of  80%. 

These  results  suggest  that  a  combination  of  high  spectral  resolution  and  texture-based  image 
segmentation  may  be  an  optimal  methodology  for  automated  classification  of  underwater  coral 
reef  imagery. 


Cl.  Materials  and  Methods 

Chi  System  Development,  Data  Acquisition,  and  Preliminary  Processing 

An  underwater,  six  band,  multispectral  camera  (MSCAM)  was  constructed  for  image  acquisition. 
The  main  components  of  the  system  included  an  Apogee  Instruments  Inc.  AP47p  CCD  camera, 
However  motorized  filter  wheel  with  space  for  six  25.4  mm  interference  filters,  a  Tamron  24mm, 
f/2.5  lens,  and  a  single  board  computer  for  controlling  the  camera  and  filter  wheel  as  well  as 
storing  images  during  acquisition.  All  of  these  components  were  housed  in  a  watertight  canister 
that  was  tethered  to  the  surface  via  a  cable  supplying  power  and  communication.  The  MSCAM 
design  is  almost  identical  to  the  NURADS  camera  (Voss  and  Chapin  2005)  except  the  latter  used 
a  fish-eye  lens  and  dome  port  to  acquire  hemispherical  images.  In  contrast,  the  MSCAM  full 
field  of  view  was  only  about  45  degrees. 

Recent  literature  on  the  hyperspectral  reflectance  of  biotic  organisms  and  substrates  associated 
with  coral  reefs  (Holden  and  LeDrew  1998;  Holden  and  LeDrew  1999;  Clark  et  al.  2000; 
Hochberg  and  Atkinson  2000;  Holden  and  LeDrew  2001;  Holden  and  LeDrew  2002;  Hochberg 
and  Atkinson  2003;  Hochberg  et  al.  2003)  was  consulted  for  guidance  the  spectral  bands 
considered  most  useful  for  optical  seabed  classification.  In  Figure  Cl  each  row  of  points 
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corresponds  to  the  bands  used  by  a  particular  study.  Solid  points  mark  band  centers,  while 
squares  and  diamonds  mark  the  locations  of  derivatives  (see  legend  for  details).  Only  six  filters 
fit  in  the  MSCAM  at  once,  so  two  sets  of  six  filters  were  used;  these  are  marked  by  the  vertical 
blue  and  red  rectangles.  Bands  centered  at  546,  568,  and  589  nm  were  used  in  both  filter  sets. 


Wavelength  (nm) 


Figure  Cl.  Proposed  spectral  bands  for  coral  reef  mapping.  See  text  for  description. 

A  full  system  calibration  was  performed  (see  Voss  and  Chapin  2005),  but  the  only  correction 
used  in  these  experiments  was  for  the  camera  lens  rolloff,  defined  as  the  relative  response  of  the 
imaging  system  as  a  function  of  location  within  the  field  of  view. 

During  data  acquisition,  the  camera  rested  on  a  tripod  in  a  downward  looking  configuration  and 
acquired  multiple  sets  of  images  with  each  of  the  six  filters.  A  panel  of  spectralon  (LabSphere, 
Inc.)  with  20  %  Lambertian  reflectance  was  placed  in  the  camera  field  of  view  during  data 
acquisition.  For  each  scene  imaged,  several  complete  sets  of  light  (shutter  open)  and  dark 
(shutter  closed)  images  were  acquired  for  each  filter.  The  AP47p  has  1024  x  1024  pixel 
resolution,  but  a  bin  factor  of  two  was  used  in  some  cases,  so  the  resulting  images  are  either  1024 
x  1024  or  512  x  512  pixels. 

After  data  acquisition  with  the  MSCAM,  images  of  each  scene  were  acquired  with  a  Sony  P-93 
in  order  to  have  traditional  three-band  (RGB)  underwater  images  for  comparison.  Images  with 
the  Sony  camera  were  also  taken  from  the  tripod  in  order  to  maintain  the  same  distance  to  the 
seabed. 

Initial  data  processing  steps  were  common  to  all  experiments.  The  first  step  in  data  processing 
was  to  correct  the  raw  image  digital  numbers  (DNs)  for  the  CCD  dark  current  and  system  rolloff 
factor  (1). 

DNC  =  (DNL  -  DNd)  /  Roff.  (1) 

DNl  and  DNd  are  the  digital  numbers  in  the  light  and  dark  current  images,  respectively.  Roff  is 
the  correction  for  system  rolloff  (Figure  C3).  DNC  are  the  corrected  digital  numbers.  The 
corrected  DNC  were  then  converted  to  reflectance  (R)  by  dividing  all  pixels  in  the  image  by  DNCS, 
(the  average  corrected  digital  numbers  observed  over  the  spectralon  panel)(2).  All  calculations  in 
(1)  and  (2)  are  performed  on  a  pixel-by-pixel  basis  for  each  band,  so  all  terms  except  for  DNCS 
are  functions  of  row,  column,  and  wavelength. 
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R  =  DNC  /  DNCS .  (2) 

Finally,  the  average  reflectance  image  (Ra)  for  each  band  was  then  computed  as  the  mean  of  all 
(N)  images  acquired  in  each  dataset. 

Cl. 2  Experiments  with  Filter  Set  1 

Experiments  with  filter  set  1  were  designed  to  test  algorithms  proposed  by  Clark  et  al.  (2000) 
and  Holden  and  LeDrew  (2001;  2002).  Four  datasets  were  acquired,  two  in  the  field  and  two  in 
saltwater  tanks  at  the  University  of  Miami  (Table  Cl).  Four  rules-based  classification  algorithms 
were  tested,  two  from  Clark  et  al.  (2000),  and  one  each  from  Holden  and  LeDrew  (2001;  2002). 
Clark  et  al.  (2000)  algorithm  A: 

•  If  the  first  derivative  at  596  nm  >  -0. 1  then  live  coral 

•  Else  if  the  first  derivative  at  596  nm  >-0.19  and  <  -0. 1  then  recently  dead  coral 

•  Else  old  dead  coral 

Clark  et  al.  (2000)  algorithm  B: 

•  If  the  second  derivative  over  5 1 5-572  nm  >  0.00253  then  live  coral 

•  Else  if  the  first  derivative  at  536  nm  >  0.07  then  recently  dead  coral 

•  Else  old  dead  coral 

Holden  and  LeDrew  (2001)  algorithm: 

•  If  the  first  derivative  between  605-625  nm  >  0.0  then  sand 

•  Else  if  the  second  derivative  at  545  nm  >  0.0  then  live  coral 

•  Else  if  the  first  derivative  between  585-605  nm  >  0.0  then  dead  coral 

Holden  and  LeDrew  (2002)  algorithm: 

•  If  the  first  derivative  between  605-625  nm  >  0.0  then  sand 

•  Else  if  the  first  derivative  between  585-605  nm  >  0.0  then  bleached  coral 

•  Else  if  the  second  derivative  at  545  nm  >  0.0  then  live  coral 

•  Else  dead  coral  or  algae 

A  fifth  algorithm  was  tested  with  decision  steps  identical  to  Clark  et  al.  (2000)  algorithm  B  but 
with  thresholds  for  each  decision  tuned  to  the  particular  scene. 


Table  Cl.  Images  for  Experiments  with  Filter  Set  1 


Shorthand 

Location 

Lighting 

N 

Tank  1 

UM  Tank 

Ambient 

1 

Pickles 

24°59.07'  N  /  80°24.97'  W 

Ambient 

6 

Tank  2 

UM  Tank 

Ambient 

5 

Shorthand  is  the  name  used  to  refer  to  the  dataset  within  the  text.  Location  is  the  decimal  latitude 
/  longitude  of  the  dataset  in  degrees  or  "UM  Tank"  for  a  saltwater  flow  though  tank  on  the 
University  of  Miami  /  RSMAS  campus.  The  sun  provided  "Ambient"  lighting;  N  is  the  number 
of  image  sets  (all  six  filters)  that  were  averaged  to  create  the  single  image  used  for  analysis. 
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Cl. 3  Experiments  with  Filter  Set  2 

Hochberg  and  Atkinson  (2000;  2003)  used  stepwise  band  selection  and  linear  discriminant 
function  analysis  to  demonstrate  that  spectral  separation  of  coral,  algae,  and  sand  reflectance 
spectra  is  possible  with  as  few  as  four  non- contiguous  narrow  wavebands.  Experiments  with 
Filter  Set  2  were  designed  to  see  if  their  suggested  bands  could  be  used  to  classify  underwater 
images. 

The  actual  discriminant  functions  used  by  Hochberg  and  Atkinson  (2000;  2003)  were  not 
reported,  so  step  one  was  to  assemble  hyperspectral  reflectance  measurements  of  live  coral, 
algae,  and  calcareous  sediment  that  we  had  made  with  underwater  point  spectrometers  (both  the 
DiveSpec  Mazel  1997  and  the  GER1500  were  used)  and  to  extract  the  portions  of  those 
measurements  corresponding  to  the  six  spectral  bands  in  our  camera.  Step  two  was  to  calculate 
the  multivariate  discriminant  functions  that  best  separated  the  spectra  belonging  to  the  three 
groups  (manoval  function  in  MATLAB  2006b,  Mathworks,  Natick,  MA).  Finally,  the 
discriminant  functions  were  applied  to  four  MSCAM  datasets  from  the  field  (Table  C2)  to  create 
the  classified  images. 


Table  C2:  Images  for  Experiments  with  Filter  Set  2 


Shorthand 

Location 

Lighting 

N 

Grecian  1 

25.10603  N/ 80.31810  W 

Ambient 

31 

Grecian  2 

25.10603  N/ 80.31810  W 

Ambient 

11 

Andros  1 

24.71900  N/ 77.77012  W 

Ambient 

16 

Andros  2 

24.71260  N/ 77.74770  W 

Ambient 

14 

Shorthand  is  the  name  used  to  refer  to  the  dataset  within  the  text.  Location  is  the  decimal  latitude 
/  longitude  of  the  dataset  in  degrees  or  "UM  Tank"  for  a  saltwater  flow  though  tank  on  the 
University  of  Miami  /  RSMAS  campus.  The  sun  provided  “Ambient”  lighting;  N  is  the  number 
of  image  sets  (all  six  filters)  that  were  averaged  to  create  the  single  image  used  for  analysis. 

Cl.  4  Experiments  with  Combined  Spectral  /  Texture  Classification 

The  method  used  for  combined  spectral  /  texture  classification  had  three  steps.  First,  a  spectral 
band  ratio  was  used  to  separate  coral  and  algae  from  other,  "background"  portions  of  the  image. 
Second,  texture  metrics  were  used  to  separate  coral  from  algae  only  for  the  part  of  the  image  that 
has  not  been  masked  as  background.  Finally,  the  output  was  smoothed  to  combine  multiple 
texture  measures  and  to  reduce  blocky  artifacts  introduced  when  small  subsections  of  the  image 
are  passed  to  the  texture  computation.  These  steps  are  described  in  detail  in  Gleason  et  al. 
(2007),  and  will  be  outlined  below. 

The  normalized  difference  ratio  of  bands  centered  at  568  and  546  nm  (ND568-546)  was  computed 
from  the  averaged  reflectance  values,  RA  (3). 

ND568-546  =  (  Ra,568  -  Ra,546  )/(  Ra,568  +  Ra,546  )•  (3) 

A  threshold  for  ND568-546  was  chosen  interactively  to  segment  pixels  containing  coral  and  algae 
from  "background"  pixels  containing  anything  else  (Table  C3).  Photosynthetic  organisms  such  as 
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corals  and  algae  tend  to  have  higher  values  of  ND568-546  than  other  organisms,  sediments,  and 
bare  rock.  All  pixels  below  the  threshold  were  set  to  zero. 

The  grey  level  co-occurrence  matrix  (GLCM)  algorithm  provided  with  MATLAB  (R2006b;  The 
MathWorks,  Inc.  Natick,  MA)  was  used  to  compute  three  texture  metrics  from  the  thresholded 
ND568-546  image.  The  MATLAB  image  processing  toolbox  documentation2  contains  details  on 
the  calculations  and  provides  two  references  (Haralick  et  al.  1973;  Haralick  and  Shapiro  1992). 
In  addition,  descriptions  of  the  algorithm  are  available  in  many  image  processing  textbooks  (e.g. 
Jain  et  al.  1995).  Thresholds  were  visually  established  for  the  other  three  texture  metrics  at  a 
level  that  maximized  the  discrimination  between  coral  and  algae  (Table  C3). 


Table  C3.  Thresholds  Used  For  Each  Algorithm  and  Dataset 


Thresholds 

Spectral 

Grey  Level  Co-occurrence  Metrics 

Dataset 

ND568-546 

Correlation 

Energy 

Homogeneity 

Tank  1 

0.10 

0.37  + 

0.12  * 

0.64  * 

Grecian  2 

0.05 

0.75  + 

0.05  * 

0.63  + 

Andros  1 

0.08 

0.35  + 

0.20  * 

0.50  + 

Tank  3 

0.05 

0.35  + 

0.20* 

0.68  * 

Thresholds  are  all  unitless.  Pixels  with  values  less  than  the  spectral  threshold  were  masked  to  a 
zero  value  (background).  Pixels  with  values  greater  than  the  spectral  threshold  were  then 
assigned  to  coral  or  algae  classes  based  on  their  value  relative  to  each  of  the  GLCM  thresholds. 
Values  less  than  the  GLCM  thresholds  marked  with  an  asterisk  (*)  were  assigned  to  the  coral 
class  and  values  greater  than  the  GLCM  thresholds  marked  with  an  asterisk  (*)  were  assigned  to 
the  algae  class.  The  reverse  is  true  for  GLCM  thresholds  marked  with  a  cross  (+). 

Output  of  the  three  GLCM  computations  was  smoothed  using  a  sliding  window  and  majority 
fdter  in  order  to  produce  a  single  output  image. 

The  accuracy  of  the  classified  images  was  assessed  by  point  counting  the  original  input  images. 
For  each  of  the  datasets,  an  analyst  used  CPCe  software  (Kohler  and  Gill  2006)  to  place  400 
random  points  over  a  false  color  image  created  from  three  of  the  six  MSCAM  bands.  The  analyst 
then  classified  each  of  the  points  as  coral,  algae  or  background. 

Results  from  point  counting  were  compared  with  the  classified  images  in  two  ways.  First,  for  an 
image-wide  comparison,  the  percentage  of  points  identified  by  the  analyst  as  coral,  algae  and 
background  was  compared  to  the  percentage  of  pixels  classified  by  the  spectral  /  texture 
algorithm  in  those  classes.  For  each  of  the  three  classes  (c),  accuracy  (Ac)  for  that  class  was  then 
computed  following  (4),  where  Pi,c  is  the  percentage  of  pixels  in  the  image  classified  in  class  C, 
and  Pa,c  is  the  percentage  of  points  identified  by  the  analyst  as  class  C. 

Ac=1-(Pi,c-Pa,c)/Pa,c.  (4) 

Second,  for  a  point-based  comparison,  the  class  for  each  of  the  400  points  identified  by  the 
analyst  was  extracted  from  the  corresponding  pixel  in  the  classified  image.  An  error  matrix  was 
then  constructed  for  each  dataset  (Congalton  and  Green  1999)  and  the  overall  accuracy  was 

2http://www.mathworks.com/access/helpdesk/help/pdf_doc/images/images_tb.pdf 
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computed  following  (5),  where  Ao  is  the  total  overall  accuracy,  Nc  is  the  number  of  points  that 
were  correctly  classified  in  class  (C),  and  Nt  is  the  total  number  of  points  counted  (400).  The 
analysis  was  repeated  both  on  the  final  classified  image,  which  had  coral,  algae,  and  background 
classes,  as  well  as  on  the  thresholded  ND568-546  image,  which  is  also  a  simple  classification  with 
just  two  classes:  coral  and  algae  combined,  and  background. 

Ao  =  ENc/Nt.  (5) 

Four  datasets  were  used  for  testing  the  spectral  /  texture  approach  (Table  IV). 


Table  C4:  Images  for  Experiments  with  Spectral  /  Texture  Classification 


Shorthand 

Location 

Lighting 

N 

Tank  1 

UM  Tank 

Ambient 

1 

Grecian  2 

25.10603  N/ 80.31810  W 

Ambient 

11 

Andros  1 

24.71900  N/ 77.77012  W 

Ambient 

16 

Tank  3 

UM  Tank 

Artificial 

5 

Shorthand  is  the  name  used  to  refer  to  the  dataset  within  the  text.  Location  is  the  decimal  latitude 
/  longitude  of  the  dataset  in  degrees  or  "UM  Tank"  for  a  saltwater  flow  though  tank  on  the 
University  of  Miami  /  RSMAS  campus.  The  sun  provided  "Ambient"  lighting;  Two  Deep  Sea 
Power  and  Light  500  W  MultiSeaLite  halogen  lamps  provided  the  “Artificial”  lighting.  Images 
with  artificial  lighting  were  acquired  at  night.  N  is  the  number  of  image  sets  (all  six  filters)  that 
were  averaged  to  create  the  single  image  used  for  analysis. 

C2.  Results  and  Discussion 


C2.1  System  Development 

System  integration  was  accomplished  during  2004  (Figure  C2).  Most  cameras  have  maximum 
response  at  the  principal  point  and  decreased  response  towards  the  edges  of  the  field  of  view;  the 
MSCAM  rolloff  was  typical  in  this  regard  (Figure  C3). 


Figure  C2.  (Left)  Interior  of  multi-spectral  camera.  (Right)  Camera  in  operation. 


149 


RC-1333  Final  Report  to  SERDP  2010 


R.P.Reid 


Figure  C3.  System  rolloff  factor  as  a  function  of  angle  from  the  center  pixel.  For  clarity,  only  two  of  the  six 
bands  are  shown:  487.1  nm  (sold)  and  589.0  nm  (dashed).  The  response  of  the  other  bands  falls  between  these 
extremes. 


C2.2  Experiments  with  Filter  Set  1 

Efforts  to  classify  imagery  from  the  multi-spectral  camera  based  on  the  schemes  tested  with  filter 
set  1  were  not  successful.  Negative  results  were  probably  due  to  two  factors.  First,  the  spectra 
used  by  Holden  and  LeDrew  (2001;  2002),  and  Clark  et  al.  (2000)  were  not  corrected  for  water 
column  attenuation,  which  is  significant  even  at  very  short  distances.  Second,  the  classes 
investigated  by  Holden  and  LeDrew  (2001;  2002),  and  Clark  et  al.  (2000)  include  only  live, 
bleached,  and  dead  coral,  and  sand.  Real  images  include  additional  classes  (algae,  silt  substrate, 
reflectance  standard  etc..)  that  are  not  accommodated  in  the  Holden  and  LeDrew  (2001;  2002), 
and  Clark  et  al.  (2000)  schemes. 

Results  from  only  the  "Tank  1"  dataset  are  presented  here  (Figures  C4,  C5),  but  they  were 
similar  for  the  other  three  images. 


Figure  C4.  False  color  600,  589,  568  nm  RGB  image  of  the  "Tank  1"  dataset  (Far  left).  Clark  2000  A,  B,  and 
tuned  classifications  for  this  dataset  (Center  left,  center  right,  far  right,  respectively).  In  the  classified  images 
white  =  live  coral;  red  =  recently  dead  coral;  blue  =  old  dead  coral. 
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Halcfcft  LeDrew  200 1  diuMation  Holden  LeDrew  2002  dtenfflcftfon 


Figure  C5.  False  color  600,  589, 568  nm  RGB  image  of  the  "Tank  1"  dataset  (Far  left).  Holden  LeDrew  (2001) 
classification  for  this  dataset  (Center;  white  =  sand;  red  =  live  coral;  blue  =  old  dead  coral;  black  = 
unclassified).  Holden  LeDrew  (2002)  classification  for  this  dataset  (Right;  white  =  sand;  red  =  live  coral;  blue 
=  bleached  coral;  black  =  algae/rubble). 


C2.3  Experiments  with  Filter  Set  2 

The  discriminant  functions  (DFs)  computed  from  our  spectra  appeared  similar  to  those  calculated 
by  Hochberg  et  al.  (2000)  when  applied  to  the  training  data  (the  point  spectrometer  data;  Figure 
C6).  The  overall  variance  along  each  DF  axis  was  similar,  as  was  the  ability  of  the  DFs  to 
discriminate  coral,  algae,  and  sand. 


tJ - J - i  & - S - TO  -2  0  2  4  6  8  10  12  14  16  18  '  -20  *“  -15  -10-5  0* 


DF  1  DF  1  DF  1 


Figure  C6.  Discriminant  functions  one  (DF1)  and  two  (DF2)  computed  from  our  full-spectrum  data  base 
(left),  from  our  partial-spectrum  data  base  (center),  and  from  Hochberg  and  Atkinson's  (2003)  database  for 
their  hypothetical  CRESPO  sensor.  In  all  cases  red  points  are  for  spectra  of  live  corals,  green  points  for 
spectra  of  algae,  and  blue  points  for  spectra  of  sediment.  Note  that  the  sign  of  DF1  is  flipped  between  our  data 
and  Hochberg  and  Atkinson  (2003),  but  the  ranges  of  both  DF1  and  DF2  are  approximately  the  same. 


When  the  discriminant  functions  were  applied  to  the  multispectral  images  (Figure  C7),  in  all 
cases  the  ranges  (max-min  values)  of  both  DF1  and  DF2  were  much  larger  in  the  MSCAM  data 
than  they  had  been  in  the  spectrometer  data  (top  row  of  Figure  C7).  This  suggests  that  there  is 
additional  variance  in  the  images  not  accounted  for  by  the  model.  Displaying  the  projected  data 
as  images,  with  DF1  in  red  and  DF2  in  green  (bottom  row  of  Figure  Cl),  reveals  that  there  are 
structured  patterns  related  to  the  geometry  of  the  scene,  not  the  benthic  cover.  The  conclusion  is 
that  geometry  is  contributing  at  least  as  much  to  the  variance  of  the  scene  as  variations  in  the 
bottom  reflectance. 
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Figure  Cl.  Four  MSCAM  scenes  acquired  with  filter  set  two.  Top  row:  results  of  applying  the  discriminant 
functions  to  the  image  data  for  each  scene.  Note  the  range  of  the  DF1  and  DF2  axes  are  much  larger  than  in 
Fig.  6.  Bottom  row:  DF1  displayed  in  red  and  DF2  displayed  in  green.  Note  variations  in  color  related  to  the 
structure  of  the  scene  and  not  just  the  benthic  cover;  this  is  particularly  noticeable  in  the  middle  two  images. 


C2.4  Experiments  with  Combined  Spectral /  Texture  Classification 

False  color  RGB  images  of  the  MSCAM  datasets  as  well  as  the  resulting  classified  images  are 
shown  for  all  four  datasets  in  Figure  C8. 

The  classified  images  agree  well  with  the  point  count  data  when  considering  image-wide 
accuracy,  which  assesses  estimates  of  the  total  percentage  of  each  class  present  in  the  scene.  The 
image-wide  accuracy,  defined  by  (4),  was  greater  than  70%  for  11  of  12  classes  and  greater  than 
75%  for  9  of  12  classes  (Table  C5). 

The  classified  images  also  agree  well  with  the  point  count  data  when  considered  on  a  point-by- 
point  basis.  The  overall  accuracies,  as  computed  from  error  matrices  using  (5),  were  greater  than 
80%  for  three  of  the  four  datasets  (Table  C5). 

Table  C5:  Image-Wide  Accuracy 


Dataset 

Coral 

Algae 

Background 

Tank  1 

74% 

71  % 

98% 

Grecian 

77% 

96% 

77% 

Andros 

78% 

11  % 

81  % 

Tank  2 

94% 

93% 

98% 

Image-wide  accuracy  was  computed  using  (4)  for  each  dataset. 


Table  C6:  Point-Based  Overall  Accuracy 


Dataset 

Overall  Accuracy 

Tank  1 

86% 

Grecian 

67% 
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Andros 

84% 

Tank  2 

87% 

Overall  accuracy  was  computed  using  (5)  for  each  dataset. 


Figure  C8.  Summary  of  results  for  combined  spectral  /  texture  classification.  Top  row:  False  color  images 
composed  of  three  multispectral  camera  bands  (589,  548,  487  nm)  displayed  in  RGB.  Bottom  row:  Final 
smoothed  classified  images  with  three  classes:  coral  (red),  algae  (green),  background  (black).  The  datasets  are 
placed  in  columns  in  the  same  order  as  Table  C5:  Tank  1,  Grecian,  Andros,  Tank  2,  from  left  to  right. 


C3.  Conclusions  and  Implications  for  Future  Research/Implementation 

As  mentioned  in  Section  C3,  initial  experiments  testing  whether  the  spectral  bands  suggested  in 
(Holden  and  LeDrew  1998;  Holden  and  LeDrew  1999;  Clark  et  al.  2000;  Hochberg  and  Atkinson 
2000;  Holden  and  LeDrew  2001;  Holden  and  LeDrew  2002;  Hochberg  and  Atkinson  2003; 
Hochberg  et  al.  2003)  could  be  used  for  automated  classification  of  underwater  imagery  were  not 
successful.  The  results  presented  above,  indicate,  however,  that  high  spectral  resolution  may 
nevertheless  be  useful  for  underwater  image  classification  because  such  data  can  greatly  simplify 
subsequent  processing  by  a  texture  classification  algorithm. 

High  spectral  resolution  is  an  improvement  over  coarse  spectral  bands  for  classifying  underwater 
imagery;  the  evidence  for  this  is  the  ability  of  ND568-546  to  more  accurately  segment  out  coral  + 
algae  than  color  matching  with  standard  underwater  photography. 

Narrow  spectral  bands  alone  are  not  a  complete  solution  for  automated  underwater  classification 
of  reef  images,  however.  Segmenting  coral  and  algae,  together  as  one  class,  from  the  background 
is  not  a  useful  end  in  itself.  On  the  other  hand,  segmenting  coral  and  algae  with  a  narrow  spectral 
band  ratio  was  useful  in  this  study  when  combined  with  image  texture  measures.  The  evidence 
for  this  was  that  thresholds  of  texture  images  computed  with  the  GLCM  algorithm  were  able  to 
separate  coral  and  algae  after  the  background  had  been  identified  with  the  narrow  band  ratio,  but 
they  could  not  do  so  without  the  prior  application  of  the  narrow  band  ratio. 
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The  approach  followed  here  was  semi-automated  in  the  sense  that  some  user  intervention  was 
required  to  select  the  four  thresholds  that  need  tuning  for  each  dataset.  Although  a  fully 
automated  system  would  be  ideal,  the  semi-automated  approach  was  still  faster  than  point 
counting,  and  there  are  at  least  three  reasons  to  expect  additional  progress  could  be  made  toward 
a  fully  automated  system,  as  follows: 

First,  the  range  of  thresholds  to  search  was  fairly  small.  The  thresholds  for  ND568-546,  GLCM 
Correlation,  and  GLCM  Homogeneity  varied  by  about  a  factor  of  two.  It  would  be  easy  to  create 
an  interface  to  facilitate  threshold  picking  so  that  many  images  could  be  analyzed  quickly. 

Second,  no  theoretical  work  has  been  performed  on  the  texture  properties  of  reef  benthos  that  is 
particularly  relevant  for  guiding  the  choice  of  GLCM  parameters.  This  differs  significantly  from 
the  situation  for  spectral  parameters,  for  which  a  number  of  previous  studies  have  suggested  that 
wavelength  regions  around  546  and  568  nm  were  good  for  discriminating  corals,  algae,  and 
substrate.  Upon  investigation,  it  may  be  possible  to  find  better  values  for  the  GLCM  block  size, 
offset,  and  smoothing  parameters  than  the  nominal  values  used  above. 

Third,  the  GLCM  algorithm  was  chosen  simply  for  convenience  due  to  its  existing 
implementation  in  MATLAB.  Better  results  may  be  expected  with  more  sophisticated  texture 
algorithms,  such  as  those  being  pursued  by  (Konotchick  et  al.  2006;  Pizarro  et  al.  2006;  Mehta  et 
al.  2007),  but  the  point  is  that  regardless  of  the  texture  algorithm  used,  acquiring  data  in  narrow 
spectral  bands  is  likely  to  improve  the  results. 

High  spectral  resolution  was  an  improvement  over  coarse  spectral  bands  for  classifying 
underwater  imagery,  but  narrow  spectral  bands  alone  were  not  a  complete  solution  for  automated 
classification  of  reef  images.  The  important  result  was  that  acquiring  underwater  imagery  in 
narrow  spectral  bands  and  pre-processing  the  data  with  a  band  ratio  greatly  simplified  texture 
classification.  Texture  classification  became  easy  enough,  in  fact,  that  the  GLCM  approach, 
which  on  its  own  was  not  useful  for  segmenting  these  images,  was  found  to  produce  reasonable 
results  when  combined  with  ND568-546- 

The  potential  for  full  automation  of  underwater  high  spectral  resolution  imagery  is  promising 
and  will  benefit  from  three  future  areas  of  research:  1)  careful  correction  for  scene  geometry,  2) 
combination  of  spectral  and  more  sophisticated  texture  algorithms,  3)  additional  research  on  the 
texture  properties  of  corals,  algae,  other  organisms  and  the  substrate  associated  with  coral  reef 
environments. 
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Appendix  2.  Supporting  Data:  Landscape  Mosaics 

A  copy  of  each  final  landscape  mosaic  created  during  this  project  will  be  included  in  the  DVD  of 
supporting  data. 

Appendix  3.  Supporting  Data:  Landscape  Mosaic  Creation  Manual 

The  complete  Landscape  Creation  Manual  will  be  provided  in  the  DVD  of  supporting  data. 

Appendix  4.  Supporting  Data:  Landscape  Mosaic  Creation  Software 

The  Landscape  Mosaic  Creation  Software  Package  including  the  external  viewer  application  and 
demonstration  images  will  be  included  as  a  separate  DVD  of  supporting  data. 

Appendix  5.  Supporting  Data:  3D  mosaicing 

The  3D  point  and  click  software,  instructions  and  sample  images  are  all  included  in  a  DVD  of 
supporting  data. 


Appendix  6.  Supporting  Data:  Multispectral  Data 

Supporting  data  and  software  are  provided  on  a  DVD  with  this  report.  Top  level  directories  are 
divided  in  four  categories:  data,  calibration,  software,  and  analysis  (Table  Cl).  Matlab  software 
(the  MathWorks,  Natick  MA)  will  be  required  to  access  many  of  these  data  files. 

"Data"  directories  correspond  to  the  "shorthand"  columns  of  Tables  Cl,  C2,  C3,  C4.  All  of  the 
data  directories  have  subdirectories  named  "ccdimages"  and  "digital_camera".  Some  of  the  data 
directories  also  contain  additional  subdirectories  "ac9"  or  "divespec".  The  "ccdimages" 
directories  contain  a  matlab  data  file  ("*.mat")  that  has  the  averaged  reflectance  image  for  that 
dataset  in  a  variable  named  "Irefmean"  or  "Irefavg"  and  the  variance  of  the  reflectance  image  in 
a  variable  named  "Irefvar".  Irefmean  is  an  average  of  the  N  replicate  images  acquired  for  that 
dataset,  where  N  is  listed  in  Tables  Cl,  C2,  C3,  C4.  The  "digital_camera"  subdirectory  contains 
hand-held  underwater  photographs  of  the  scene  for  comparison  with  the  MSCAM  images. 
Usually  there  are  additional  photographs  in  the  "digitalcamera"  subdirectory  providing  close- 
ups  of  various  organisms  in  the  scene  or  images  documenting  the  experimental  set  up.  The 
"divespec"  directories,  when  present,  contain  reflectance  measurements  made  with  underwater 
spectroradiometers  for  that  dataset.  The  divespec  files  are  named  MDDHHMMS.AYY  where  M 
=  month(A-L),  DD  =  day(00-31),  HH  =  hour(00-23),  MM  =  minute(00-59),  S  =  1/10  of  second 
(0-9),  A  is  the  instrument  ID  (always  A  for  our  instrument),  and  YY  is  the  two  digit  year.  The 
format  of  the  divepsec  files  is  ASCII  text  with  a  short  header  followed  by  two  columns: 
wavelength  (nm)  and  reflectance  factor  (0-1).  The  AC-9  directories  contain  AC-9  raw  data  files 
(*.dat),  processed  data  files  (*.mat)  and  plots  (*.jpg),  as  well  as  the  matlab  scripts  used  for 
processing  (*.m).  The  AC-9  raw  data  format  is  described  in  the  file  "ac9usersguide.pdf, 
contained  on  the  DVD.  The  processed  ac-9  files  contain  structures  at  various  levels  of 
processing.  The  one  of  interest  is  named  "wmedcorscat"  (the  highest  level  of  processing).  It 
contains  water  column  absorption  (a)  and  attenuation  (c)  values  in  the  field  "medianpluswater". 
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The  "calibration"  directories  contain  rolloff  correction  images  computed  on  two  dates.  On  28 
September  2004  we  performed  a  calibration  averaging  all  six  filters  (filter  set  1).  The  result  is 
contained  in  the  matlab  data  file  "rolloff_040928.mat".  It  is  a  matrix  the  same  size  as  the  images 
that  were  multipled  by  the  raw  data  to  generate  the  reflectance  images.  On  7  April  2007  a  second 
calibration  was  performed  (filter  set  2).  In  this  set,  a  different  rolloff  image  was  generated  for 
each  filter  (they  are  slightly  different).  The  calibration  images  should  not  be  needed  for  further 
processing,  but  are  provided  as  a  reference. 

The  "software"  directory  contains  Matlab  functions  used  for  processing.  The  files  in  "ac9"  were 
used  to  calibrate  the  raw  ac-9  data.  The  file  "loaddivespec.m"  was  used  to  read  the  divespec  data 
files.  The  scripts  in  the  "msc"  directory  were  used  to  process  and  display  the  MSCAM  images 
themselves.  There  are  descriptions  in  the  comment  section  at  the  beginning  of  each  of  these  files 
describing  their  functionality. 

The  "analysis"  directory  contains  output  from  the  three  experiments  described  in  the  report.  Files 
under  "exp_filtersetl_clark_holden_ledrew"  were  used  to  test  the  filter  set  one  algorithms.  Files 
under  "exp_filterset2_discriminant_analysis"  were  used  to  test  the  filter  set  two  algorithms.  Files 
under  "  expspectexture"  were  used  to  test  the  combined  spectral  threshold  and  texture 
processing  algorithm.  Generally,  these  files  contain  a  matlab  script  ("*.m")  that  controlled  the 
processing,  output  images  ("*.jpg")  and,  in  some  cases,  intermediate  data  files  ("*.mat"). 


Appendix  7.  Supporting  Data:  Landscape  Mosaic  Surveying  Technology 

Requirements 


Software  Requirements 

The  mosaic  creation  requires  the  following  software 

•  SERDP  Mosaic  software  deliverable  disk 

•  Microsoft(R)  Windows  XP  (64  bit)  professional  SP2 

•  MathWorks(R)  Matlab  version  R2009A  (later  versions  are  not  recommended  and  may 
have  compatibility  problems)  with  the  following  toolboxes:  Optimization,  statistics, 
image  processing 

•  Camera  calibration  toolbox  for  Matlab  (freely  available  at 
http://www.vision.caltech.edu/bougueti/calib  doc/ ) 


The  following  software  is  highly  recommended.  It  may  be  replaced  with  other  packages  that 
provide  the  same  functionality.  However,  this  manual  provides  detailed  examples  using  these 
packages. 

2.  Adobe(R)  Premiere  6.0  (if  using  Standard  Definition  Video) 

3.  Sony(R)  Vegas  Movie  Studio  HD  (if  using  High  Definition  Video) 

4.  JASC(R)  Paint  Shop  Pro  (any  version) 

5.  Videolan(R)  VLC  media  player  (freely  available  at  http://www.videolan.org/vlc/) 
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Software  Installation 

1.  Install  Microsoft(R)  Windows  XP,  MathWorks(R)  Matlab,  Adobe(R)  Premiere  6.0, 
Sony(R)  Vegas  Movie  Studio  HD,  JASC(R)  Paint  Shop  Pro  and  Videolan(R)  VLC 
according  to  product  instructions. 

2.  Install  the  camera  calibration  toolbox. 

3.  Copy  the  contents  of  the  SERDP  Mosaic  software  deliverable  disk  into  directory 
c:\users\serdp\  .  After  copying,  there  should  be  a  subdirectory  c:\users\serdp\ 
nunomosaics  with  several  subdirectories,  that  contains  the  mosaicing  code.  There  will  be 
also  a  directory  c:\users\serdp\test  that  contains  test  data. 

4.  Inside  MATLAB,  open  menu  File->Set  Path,  and  click  on  "add  with  subfolders' . 

5.  Select  the  c:\users\serdp\nunomosaics  directory  and  press  ‘ save ’. 


Acquisition  Hardware 

The  following  acquisition  hardware  was  used  to  produce  the  examples  shown  in  this  manual,  and 
for  the  results  in  the  project  report. 

-  Sony  TRV900  digital  video  camera  (standard  definition) 

-  Sony  HVR  -  A1U  digital  video  camera  (high  definition) 

-  Nikon  D200  lOMPixel  digital  still  camera  with  AF-S  Nikkor  18-70mm  ED  lens 

-  KVH  Industries  Cl 00  Compass  Engine 

-  Precision  Voltage  to  Frequency  Converter  LM23 1  manufactured  by  National  Semiconductor 
and  we  implemented  the  circuit  in  figure  1  "Simple  Stand-Alone  Voltage-To-Frecuency 
Converter  with  +-0.03%  Typical  Linearity  (f=10Hz  to  11  KHz)  to  convert  compass  output 
voltage  into  a  frequency  modulated  audio  signal,  recorded  by  one  of  the  HDV  camera  audio 
channels.  Datasheet  for  Precision  Voltage  to  Frequency  Converter  LM231,  2006, 

National  Semiconductor,  www.national.com/ds/LM/*LM231*.pdf 

There  is  no  requirement  for  specific  camera  model,  image  resolution  or  type  of  lenses  to  use  the 
mosaic  creation  and  analysis  tools  covered  by  this  manual.  However,  the  camera  needs  to  be 
calibrated  (detailed  in  the  next  section)  in  order  to  eliminate  the  effect  of  lens  and  housing 
distortions. 

The  heading  sensor  is  composed  of  a  fluxgate  compass  with  a  precision  voltage  frequency 
converter.  The  compass  interface  electronic  circuit  was  designed,  tested  and  calibrated  for  this 
particular  project.  The  heading  information  is  provided  by  a  KVH  fluxgate  electronic  compass 
that  measures  the  Earth’s  magnetic  field  and  produces  an  analog  voltage  output  proportional  to 
the  compass  heading.  This  voltage  is  converted  to  a  frequency  in  the  audio  range  and  stored  in 
the  audio  channel  #1  of  the  high  definition  video  camera.  The  heading  sensor  and  converter 
produce  an  audio  signal  with  the  following  relevant  characteristics: 

-  Frequency  at  0  degree  heading  :  236Hz 

-  Frequency  at  360  degree  heading:  441 1Hz 

-  Linear  frequency  response 

-  The  heading  to  frequency  transfer  function  is  freq(Hz)  =  1 1.5989  x  heading(deg)  +  236 
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The  software  can  use  any  other  heading  sensor  that  has  been  adjusted  to  have  a  similar  frequency 
transfer  function. 

Computing  hardware  and  typical  disk  and  CPU  usage 

The  minimum  hardware  configuration  is 

•  Pentium4  2GHz 

•  3  GB  RAM 

•  30  GB  free  disk  space  per  survey 

The  recommended  configuration  is 

•  Pentium  Core2  Quad  2.8GHz  or  higher 

•  8  GB  RAM 

•  NVIDIA  CUDA(tm)  enabled  graphics  card 

•  30  GB  free  disk  space  per  survey 


The  following  tables  show  disk  usage  and  execution  times  for  a  typical  mosaic  creation  job.  The 
survey  covered  250  sq  meters  and  was  recorded  in  30  min  High  Definition  Video.  The  mosaic 
was  created  on  a  2.8GHz  Pentium  IV  with  3GB  RAM.  The  CPU  Processing  table  refers  to  the 
computation  time.  The  Required  User  Intervention  table  refers  to  the  actual  intervention  time 
spend  by  the  user  (human  attention). 


DISK  USAGE 


Raw  Video 

Download 

Extracted 

Video  to  be 
processed 

Extracted  and 

corrected 

images 

Intermediate 
processing  files 

Final  Mosaic 
Image 

5GB 

1.5GB 

2.1  GB 

200MB 

20MB 

CPU  PROCESSING  TIME 


Video 

download 

from 

camera 

AVI 

creation 

Image 

extraction 

and 

correction 

Matching 
and  global 
optimization 

Mosaic 

rendering 

Job  Total 
Time 

0.5h 

5h 

llh 

36h 

3h 

55. 5h 

REQUIRED  USER  INTI 

ERVENTION  TIME 

Video 

download 

from 

camera 

AVI 

creation 

Image 

extraction 

and 

correction 

Matching 
and  global 
optimization 

Mosaic 

rendering 

Total 

User 

Time 

lOmin 

40min 

lOmin 

15  min 

15  min 

90min 
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Guidelines  for  handheld  image  acquisition 

This  text  contains  some  guidelines  to  assist  divers  during  handheld  image  acquisition. 

Generic  guidelines 

Camera  should  be  facing  the  floor 

-  Avoid  pan  (i.e.  sideways)  and  tilt  (i.e.  up  and  down)  rotation. 

Avoid  fast  rotation  motion  (at  the  end  of  each  swath). 

Handheld  video  cameras 

-  DV  video  camera  should  be  set  to  non-interlaced  video  mode.  Commonly  video  cameras 
operate  in  interlaced  mode,  where  each  recorded  image  contains  two  independent  half- 
resolution  frames  acquired  at  different  times  (typically  60  Hz).  This  creates  smoother 
motion  to  the  human  eye  during  playback.  However  it  ruins  the  performance  of  the 
mosaic  creation  algorithms  which  assume  each  frame  is  captured  at  a  single  instant. 
Cameras  should  be  set  to  high  shutter  speed.  Some  cameras  allow  for  manually  defining 
the  shutter  speed,  while  the  aperture  (camera  iris)  is  automatically  controlled  to  adjust  to 
the  light  conditions.  In  such  case,  this  mode  should  be  used  at  a  high  shutter  speed  (1/200 
or  higher),  as  long  as  there  is  enough  light.  By  using  high  shutter  speed,  the  motion  blur 
on  each  frame  is  reduced,  resulting  in  more  precise  motion  estimation. 

Camera  motion  should  be  slow.  This  way,  the  effects  of  motion  blur  are  reduced.  As  a 
rule  of  thumb  use  such  speed  that  an  object  entering  the  top  of  the  image  takes  at  least  5 
seconds  to  disappear  at  the  bottom  of  the  image. 

Zoom  settings  should  not  be  changed  during  acquisition.  The  motion  estimation 
algorithms  usually  assume  that  the  camera’s  focal  length  is  constant  throughout  the 
sequence.  Also,  in  order  to  allow  angular  measurements  over  the  images,  it  is  extremely 
helpful  to  have  images  of  the  calibration  grid  acquired  with  the  same  zoom  setting.  A 
typical  good  zoom  setting  is  having  the  cameras  at  the  widest  angle  (i.e.  largest  field  of 
view). 
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During  Acquisition 

Cover  the  area  using  a  double  “lawn  mower”  pattern 


First  forward  and  backwards,  and  then  sideways. 

Make  sure  you  have  noticeable  overlap  between  strips.  Do  not  forget  to  do  the  sideways 
pattern;  otherwise  the  sequence  is  unusable  for  mosaicing.  If  possible,  keep  the  relative 
camera  orientation  the  same.  Rotate  the  diver  around  the  camera  when  changing  directions 
rather  than  rotating  the  camera. 


Still  Photography  Camera 

Still  cameras  can  provide  very  high  resolution,  but  cannot  acquire  sequences  in  a  continuous  and 
smooth  way.  Not  all  mosaicing  algorithms  can  take  advantage  of  still  images.  However  the  high 
resolution  can  be  valuable  in  providing  texture  details.  The  main  guidelines  are  the  following 

Ensure  high  superposition  from  image  to  image.  Superposition  should  be  at  least  60% 
between  every  pair  of  consecutive  images. 

Stand  still  when  taking  the  photo,  especially  if  there  is  low  light  intensity. 

Whenever  possible,  keep  camera  facing  down  at  constant  distance. 


Camera  Setup  Guidelines  for  High  Definition  Video  and  10MP  Still 
Cameras 

HDV  setup: 

•  Wide  Angle  Zoom: 

>  Use  the  ZOOM  radial  button  on  the  exterior  of  the  camera  to  the  widest  possible 
angle  zoom. 

Slide  the  power  switch  down  to  CAMERA-TAPE  mode.  Touch  P-MENU  ->[MENU],  Select  the 
CAMERA  SET  menu. 
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•  Turn  off  automatic  camera  stabilization: 

>  Move  down  in  the  CAMERA  SET  menu  and  select  STEAD YSHOT. 

>  Set  STEADYSHOT  to  OFF. 

>  Then  press  [OK], 

•  Set  shutter  speed  to  1/250: 

>  Move  down  in  the  CAMERA  SET  menu  and  select  the  SHUTTR  SPEED. 

>  Select  the  MANUAL  setting  and  use  the  -/+  buttons  to  scroll  through  the  different 
options  and  select  the  shutter  speed  of  250. 

>  Then  press  [OK], 

•  Set  the  exposure  to  AUTO: 

>  Move  down  in  the  CAMERA  SET  menu  and  select  the  EXPOSURE. 

>  Select  the  AUTO  setting. 

>  Then  press  [OK], 

•  Set  white  balance  to  highest  color  temperature: 

>  Move  down  in  the  CAMERA  SET  menu  and  select  the  WHITE  BAL. 

>  Frame  a  blue  paper/object  in  front  of  the  camera,  the  paper/object  should  take  up 
the  entire  view  of  the  camera. 

>  Select  the  ONE  PUSH  option  which  looks  like  a  rectangle  over  two  symmetrical 
triangles. 

>  Touch  the  ONE  PUSH  symbol  once  again. 

>  The  ONE  PUSH  symbol  will  begin  to  flash  quickly.  When  the  white  balance  has 
been  adjusted  and  stored  in  the  memory,  the  indicator  stops  flashing. 

>  After  the  white  balanced  has  been  stored,  push  [OK] 

•  Set  white  balance  to  highest  color  temperature: 


Still  Camera  setup: 


•  Verify  the  camera  is  in 
auto  focus: 

>  The  Focus  mode 
should  be  set  at  S 
mode  on  the  exterior 
button  of  the  camera. 


•  Set  camera  to  highest  zoom: 

>  Use  the  ZOOM  RING  to  adjust  for  the  highest  zoom  (24  mm) 
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Lens  - 

An  AF-S  DX  1 8-70  mm  f/3.5-4.5G  ED  lens  is  used  in  this  manual  for  illustrative  purposes. 
The  parts  of  the  lens  are  shown  below. 

Zoom  ring 

Focal  length  scale 


Mounting  index: HR 8 
CPU  contacts:®  176 


Focus  ring:®  60  A-M  mode  switch:®  60 


The  following  settings  will  require  entering  the  cameras  menu: 

Most  shooting,  playback,  and  setup  options  can 
be  accessed  from  the  camera  menus.  To  view  the 
menus,  press  the  Q  button. 

The  multi  selector  is  used  to  navigate  through  the  camera  menus. 


Cancel 


Return  to  previous  menu 
(can  also  be  performed 
with  ©  button) 


Move  cursor  down 
Decrease  number 


Move  cursor  up 
Increase  number 


Display  sub-menu 
Make  selection  (selection  can  also 
be  made  with  ©  button  or  center 
of  multi  selector) 


Make  selection  (same  as  ©  button) 


•  Program  camera  for  1  frame  every  2  seconds: 

>  In  the  MENU  select  the  SHOOTING  MENU  highlighted  as  a  small  camera  icon. 


1 


Highlight  Intvl  Timer  Shooting  in  the  shoot¬ 
ing  menu  m  124)  and  press  the  multi  selector 
to  the  right. 
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Press  the  multi  selector  up  or  down  to  choose 
one  of  the  following  Start  options: 

’  Now:  Shooting  begins  after  a  delay  of  about  3  s 

■  Start  time:  Shooting  begins  at  Start  time 
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Press  the  multi  selector  left  or  right  to  highlight 
the  following  options  and  press  the  multi  selec¬ 
tor  up  or  down  to  change  interval  timer  set- 


^  Press  the  multi  selector  left  or  right  to  highlight 
the  following  options  and  press  the  multi  selec¬ 
tor  up  or  down  to  change  interval  timer  set¬ 
tings. 


Option 

Dtiolpttoi 

Start  tlm# 

Enter  start  time  for  interval  timer  photography  when  Start  time, 
is  selected  for  Start.  Press  multi  selector  left  or  right  to  highlight 
starting  hour  or  minute,  press  up  or  down  to  change.  Not  available 
when  Now  is  selected  fcf  Start. 

Interval 

Enter  time  between  shots  Press  multi  selector  left  or  right  to  high¬ 
light  hour,  minute,  or  second  press  up  or  down  to  change.  Notes 
that  camera  will  not  be  able  to  take  photographs  at  specified  In¬ 
terval  if  interval  is  shorter  than  shutter  speed  or  time  required  to 
record  images, 

S«b(t 

lntvl*Shots 

Choose  number  of  intervals  and  number  of  shots  taken  at  each  in¬ 
terval.  Press  multi  selector  left  or  right  to  highlight  number  of  inter- 
va  Is  or  n  um  ber  of  sh  ots,  press  up  or  down  to  c  h  a  nge.  Tota  1  n  urn  ter 
of  s  hots  th  at  wi  1 1  be  ta  ken  appears  to  right. 

Remaining 
linlvl  Shots) 

Shows  number  of  intervals  and  total  shots  remaining  in  current  in¬ 
terval  program.  This  item  can  not  he  edited. 

Start 

Choose  Off  to  adjust  settings  without  starting  interval  timer.  To  start 
interval  timer,  select  On  and  press  ©.  Shooting  will  start  at  selectee 
start  time  and  will  continue  for  specified  number  of  intervals. 

series  of  shots  will  be  taken  at  the  specified  starting  time.  Shooting  will 
continue  at  the  selected  interval  until  all  shots  have  been  taken.  If  shoot¬ 
ing  can  not  proceed  at  current  settings  (for  example,  if  a  shutter  speed  of 
&u  i  b  is  currently  selected  in  manual  exposure  mode,  or  the  starting  time 
is  less  than  one  minute  from  the  current  time),  a  warning  will  appear  and 
the  interval  timer  menu  will  be  displayed  again. 


Use  of  a  tripod  is  recommended. 


•  Set  ISO  to  800  (for  very  bright  day  and  depth  of  <  4  meters,  use  ISO  at  400) 

>  In  the  SHOOTING  MENU  select  the  ISO  SENSITIVITY  menu. 

>  Use  the  multi  selector  to  set  the  ISO  to  800  (400  for  bright  days  and  small  depths). 

•  Set  image  quality  setting  (Large;  High  Quality;  JPEG): 

>  In  the  SHOOTING  MENU  select  the  IMAGE  QUALITY  menu. 

>  Highlight  the  ‘JPEG  FINE’  option  of  the  image  quality  list  and  press  the  multi 
selector  to  the  right. 


•  Set  the  white  balance  to  10000K  color  temperature: 
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>  In  the  SHOOTING  MENU  select  the  WHITE  BALANCE  menu. 

>  Select  the  CHOOSE  COLOR  TEMP  menu. 

>  Select  the  10000K  option  and  right  click  [OK], 

•  Set  Exposure  program  to  shutter  priority  with  shutter  speed  1/320: 

>  Press  the  exterior  button  MODE  and  use  the  main  command  dial  until  the  M 
(Manual)  option  shows  up. 

Rotate  the  main  command  dial  to  choose  a  shutter  speed  of  320. 

Appendix  8.  Supporting  Data:  Action  Items 

On  the  disk  of  supporting  data  will  be  two  additional  reports  submitted  to  SERDP:  1)  an  analysis 
of  advances  in  mosaicing  and  benthic  survey  technology  accomplished  during  the  course  of 
SI1333  and  2)  a  cost  benefit  analysis  of  survey  methodologies. 
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